You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/08/23 11:23:07 UTC
[GitHub] [hive] ashish-kumar-sharma opened a new pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
ashish-kumar-sharma opened a new pull request #1419:
URL: https://github.com/apache/hive/pull/1419
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375675
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487367557
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r495127587
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string dbName,
+ 2: required string tblName,
+ 3: required string catName
Review comment:
3 of the 6 constraint has catName required hence I have to make it required. We can open new jira to make a consistent behaviour across all the constraints.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487382568
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487551559
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
It is only called when copy/snapshot of the table is taken in semantic analyser.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r486978810
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375420
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487366456
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on pull request #1419:
URL: https://github.com/apache/hive/pull/1419#issuecomment-679876474
@vineetgarg02 Can please review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487538671
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5664,69 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
- }
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName)
+ throws HiveException, NoSuchObjectException {
try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
+ return getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName, getDefaultCatalog(conf)));
} catch (NoSuchObjectException e) {
Review comment:
Will NoSuchObjectException comes here? From the interface, it sounds not. Remove the handling here too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh merged pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh merged pull request #1419:
URL: https://github.com/apache/hive/pull/1419
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r486978810
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487382545
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487540360
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getTableConstraints(this.getDbName(), this.getTableName(), true, true);
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
Review comment:
Callers are assuming tableConstraintsInfo won't be null after invoking this method but it is not if there is an exception.
Can initialize it with default constructor in this flow.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched = false;
Review comment:
Shall remove this flag as we can check tableConstraintsInfo != null instead. Btw, we need this flag if we use default constructor in exception flow of getTableConstraintsInfo() method.
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
I can still see it here.
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string dbName,
+ 2: required string tblName,
+ 3: required string catName
Review comment:
catName can be optional.
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9330,6 +9330,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all constraints attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
Review comment:
This method also throws MetaException.
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -2502,6 +2520,9 @@ PartitionsResponse get_partitions_req(1:PartitionsRequest req)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
CheckConstraintsResponse get_check_constraints(1:CheckConstraintsRequest request)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
+ // All table constrains
+ AllTableConstraintsResponse get_all_table_constraints(1:AllTableConstraintsRequest request)
Review comment:
Here it shows, this method returns NoSuchObjectException but actually not. I think, the implementation should return this exception and handle in HMS client as well.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Need to understand from the caller on the usage and add appropriate comment to mark this behavior. Deep-copy would add additional overhead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r477043530
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##########
@@ -11286,6 +11287,20 @@ private String getPrimaryKeyConstraintName(String catName, String dbName, String
return notNullConstraints;
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
+ throws MetaException {
+ debugLog("Get all table constraints for the table - " + catName + "."+ db_name+"."+tbl_name + " in class ObjectStore.java");
+ SQLAllTableConstraints sqlAllTableConstraints = new SQLAllTableConstraints();
Review comment:
https://issues.apache.org/jira/browse/HIVE-24062
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r479678899
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,382 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.SQLCheckConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.List;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
+public class TestGetAllTableConstraints
+ extends MetaStoreClientTest {
+ private static final String OTHER_DATABASE = "test_constraints_other_database";
+ private static final String OTHER_CATALOG = "test_constraints_other_catalog";
+ private static final String DATABASE_IN_OTHER_CATALOG = "test_constraints_database_in_other_catalog";
+ private final AbstractMetaStoreService metaStore;
+ private IMetaStoreClient client;
+ private Table[] testTables = new Table[3];
+ private Database inOtherCatalog;
+
+ public TestGetAllTableConstraints(String name, AbstractMetaStoreService metaStore) throws Exception {
+ this.metaStore = metaStore;
+ }
+
+ @Before
+ public void setUp() throws Exception {
+ // Get new client
+ client = metaStore.getClient();
+
+ // Clean up the database
+ client.dropDatabase(OTHER_DATABASE, true, true, true);
+ // Drop every table in the default database
+ for (String tableName : client.getAllTables(DEFAULT_DATABASE_NAME)) {
+ client.dropTable(DEFAULT_DATABASE_NAME, tableName, true, true, true);
+ }
+
+ client.dropDatabase(OTHER_CATALOG, DATABASE_IN_OTHER_CATALOG, true, true, true);
+ try {
+ client.dropCatalog(OTHER_CATALOG);
+ } catch (NoSuchObjectException e) {
+ // NOP
+ }
+
+ // Clean up trash
+ metaStore.cleanWarehouseDirs();
+
+ new DatabaseBuilder().setName(OTHER_DATABASE).create(client, metaStore.getConf());
+
+ Catalog cat =
+ new CatalogBuilder().setName(OTHER_CATALOG).setLocation(MetaStoreTestUtils.getTestWarehouseDir(OTHER_CATALOG))
+ .build();
+ client.createCatalog(cat);
+
+ // For this one don't specify a location to make sure it gets put in the catalog directory
+ inOtherCatalog = new DatabaseBuilder().setName(DATABASE_IN_OTHER_CATALOG).setCatalogName(OTHER_CATALOG)
+ .create(client, metaStore.getConf());
+
+ testTables[0] = new TableBuilder().setTableName("test_table_1").addCol("col1", "int").addCol("col2", "int")
+ .addCol("col3", "boolean").addCol("col4", "int").addCol("col5", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[1] = new TableBuilder().setDbName(OTHER_DATABASE).setTableName("test_table_2").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ testTables[2] = new TableBuilder().inDb(inOtherCatalog).setTableName("test_table_3").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ // Reload tables from the MetaStore
+ for (int i = 0; i < testTables.length; i++) {
+ testTables[i] =
+ client.getTable(testTables[i].getCatName(), testTables[i].getDbName(), testTables[i].getTableName());
+ }
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ try {
+ if (client != null) {
+ try {
+ client.close();
+ } catch (Exception e) {
+ // HIVE-19729: Shallow the exceptions based on the discussion in the Jira
+ }
+ }
+ } finally {
+ client = null;
+ }
+ }
+
+ @Test
+ public void noConstraints() throws TException {
+ Table table = testTables[0];
+
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+ Assert.assertTrue(fetched.getDefaultConstraints().isEmpty());
+ Assert.assertTrue(fetched.getNotNullConstraints().isEmpty());
+ Assert.assertTrue(fetched.getPrimaryKeys().isEmpty());
+ Assert.assertTrue(fetched.getUniqueConstraints().isEmpty());
+ }
+
+ @Test
+ public void fewPresentWithMultipleConstraints() throws TException {
+ Table table = testTables[0];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col2 with not null constraint in default catalog and database;
+ String nnCol2ConstraintName = "col2_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(nnCol2ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol3 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol3);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
+ Assert.assertEquals(1, fetched.getPrimaryKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getPrimaryKeys().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getPrimaryKeys().get(0).getTable_name());
+ Assert.assertEquals("col1", fetched.getPrimaryKeys().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getPrimaryKeys().get(0).getKey_seq());
+ Assert.assertEquals(pkConstraintName, fetched.getPrimaryKeys().get(0).getPk_name());
+ Assert.assertTrue(fetched.getPrimaryKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getPrimaryKeys().get(0).getCatName());
+
+ // Assert unique constraint
+ Assert.assertEquals(1, fetched.getUniqueConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getUniqueConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getUniqueConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getUniqueConstraints().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getUniqueConstraints().get(0).getKey_seq());
+ Assert.assertEquals(uniqueConstraintName, fetched.getUniqueConstraints().get(0).getUk_name());
+ Assert.assertTrue(fetched.getUniqueConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getUniqueConstraints().get(0).getCatName());
+
+ // Assert Default constraint
+ Assert.assertEquals(1, fetched.getDefaultConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getDefaultConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getDefaultConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getDefaultConstraints().get(0).getColumn_name());
+ Assert.assertEquals("false", fetched.getDefaultConstraints().get(0).getDefault_value());
+ Assert.assertEquals(defaultConstraintName, fetched.getDefaultConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getDefaultConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getDefaultConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint
+ Assert.assertEquals(2, fetched.getNotNullConstraintsSize());
+
+ // Assert Not Null constraint for col2
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getNotNullConstraints().get(0).getColumn_name());
+ Assert.assertEquals(nnCol2ConstraintName, fetched.getNotNullConstraints().get(0).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint for col3
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(1).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(1).getTable_name());
+ Assert.assertEquals("col3", fetched.getNotNullConstraints().get(1).getColumn_name());
+ Assert.assertEquals(nnCol3ConstraintName, fetched.getNotNullConstraints().get(1).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(1).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(1).getCatName());
+
+ // Check constraints which is not present in table
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+
+ }
+
+ @Test
+ public void allConstraintsPresent() throws TException {
+ Table table = testTables[0];
+ Table parentTable = testTables[1];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col2 with not check constraint in default catalog and database;
+ String ccCol2ConstraintName = "col2_check";
+ List<SQLCheckConstraint> cc =
+ new SQLCheckConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(ccCol2ConstraintName)
+ .setCheckExpression("= 5").build(metaStore.getConf());
+ client.addCheckConstraint(cc);
+
+ // Set col1 of parent table to PK and Set Col4 of table to FK
+ String parentPkConstraintName = "parentpk";
+ List<SQLPrimaryKey> parentPk =
+ new SQLPrimaryKeyBuilder().onTable(parentTable).addColumn("col1").setConstraintName(parentPkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(parentPk);
+ String fkConstraintName = "fk";
+ List<SQLForeignKey> fk =
+ new SQLForeignKeyBuilder().fromPrimaryKey(parentPk).onTable(table).setConstraintName(fkConstraintName)
+ .addColumn("col4").build(metaStore.getConf());
+ client.addForeignKey(fk);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
+ Assert.assertEquals(1, fetched.getPrimaryKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getPrimaryKeys().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getPrimaryKeys().get(0).getTable_name());
+ Assert.assertEquals("col1", fetched.getPrimaryKeys().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getPrimaryKeys().get(0).getKey_seq());
+ Assert.assertEquals(pkConstraintName, fetched.getPrimaryKeys().get(0).getPk_name());
+ Assert.assertTrue(fetched.getPrimaryKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getPrimaryKeys().get(0).getCatName());
+
+ // Assert unique constraint
+ Assert.assertEquals(1, fetched.getUniqueConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getUniqueConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getUniqueConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getUniqueConstraints().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getUniqueConstraints().get(0).getKey_seq());
+ Assert.assertEquals(uniqueConstraintName, fetched.getUniqueConstraints().get(0).getUk_name());
+ Assert.assertTrue(fetched.getUniqueConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getUniqueConstraints().get(0).getCatName());
+
+ // Assert Default constraint
+ Assert.assertEquals(1, fetched.getDefaultConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getDefaultConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getDefaultConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getDefaultConstraints().get(0).getColumn_name());
+ Assert.assertEquals("false", fetched.getDefaultConstraints().get(0).getDefault_value());
+ Assert.assertEquals(defaultConstraintName, fetched.getDefaultConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getDefaultConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getDefaultConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint
+ Assert.assertEquals(1, fetched.getNotNullConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getNotNullConstraints().get(0).getColumn_name());
+ Assert.assertEquals(nnCol3ConstraintName, fetched.getNotNullConstraints().get(0).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(0).getCatName());
+
+ // Assert check constraint
+ Assert.assertEquals(1, fetched.getNotNullConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getCheckConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getCheckConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getCheckConstraints().get(0).getColumn_name());
+ Assert.assertEquals("= 5", fetched.getCheckConstraints().get(0).getCheck_expression());
+ Assert.assertEquals(ccCol2ConstraintName, fetched.getCheckConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getCheckConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getCheckConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getCheckConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getCheckConstraints().get(0).getCatName());
+
+ // Assert foreign key
+ Assert.assertEquals(1, fetched.getForeignKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getForeignKeys().get(0).getFktable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getForeignKeys().get(0).getFktable_name());
+ Assert.assertEquals("col4", fetched.getForeignKeys().get(0).getFkcolumn_name());
+ Assert.assertEquals(parentTable.getDbName(), fetched.getForeignKeys().get(0).getPktable_db());
+ Assert.assertEquals(parentTable.getTableName(), fetched.getForeignKeys().get(0).getPktable_name());
+ Assert.assertEquals("col1", fetched.getForeignKeys().get(0).getPkcolumn_name());
+ Assert.assertEquals(1, fetched.getForeignKeys().get(0).getKey_seq());
+ Assert.assertEquals(parentPkConstraintName, fetched.getForeignKeys().get(0).getPk_name());
+ Assert.assertEquals(fkConstraintName, fetched.getForeignKeys().get(0).getFk_name());
+ Assert.assertTrue(fetched.getForeignKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getForeignKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getForeignKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getForeignKeys().get(0).getCatName());
+
+ }
Review comment:
We will not get NoSuchObjectException because in the code follow NoSuchObjectException is replaced by MetaException
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375367
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487366996
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma edited a comment on pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma edited a comment on pull request #1419:
URL: https://github.com/apache/hive/pull/1419#issuecomment-679876474
@vineetgarg02 Can you please review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r479632452
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
Review comment:
https://issues.apache.org/jira/browse/HIVE-22782
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,382 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.SQLCheckConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.List;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
Review comment:
Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r486978810
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r495132656
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5664,69 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
- }
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName)
+ throws HiveException, NoSuchObjectException {
try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
+ return getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName, getDefaultCatalog(conf)));
} catch (NoSuchObjectException e) {
Review comment:
I have added NoSuchObjectException in getAllTableConstraints() So we need to handle this here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375471
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r479632452
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
Review comment:
https://issues.apache.org/jira/browse/HIVE-24091
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487366939
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375660
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487382720
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] adesh-rao commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
adesh-rao commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r477042339
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,145 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
+public class TestGetAllTableConstraints extends MetaStoreClientTest {
+ private static final String OTHER_DATABASE = "test_constraints_other_database";
+ private static final String OTHER_CATALOG = "test_constraints_other_catalog";
+ private static final String DATABASE_IN_OTHER_CATALOG = "test_constraints_database_in_other_catalog";
+ private final AbstractMetaStoreService metaStore;
+ private IMetaStoreClient client;
+ private Table[] testTables = new Table[3];
+ private Database inOtherCatalog;
+
+ public TestGetAllTableConstraints(String name, AbstractMetaStoreService metaStore) throws Exception {
+ this.metaStore = metaStore;
+ }
+ @Before
+ public void setUp() throws Exception {
+ // Get new client
+ client = metaStore.getClient();
+
+ // Clean up the database
+ client.dropDatabase(OTHER_DATABASE, true, true, true);
+ // Drop every table in the default database
+ for(String tableName : client.getAllTables(DEFAULT_DATABASE_NAME)) {
+ client.dropTable(DEFAULT_DATABASE_NAME, tableName, true, true, true);
+ }
+
+ client.dropDatabase(OTHER_CATALOG, DATABASE_IN_OTHER_CATALOG, true, true, true);
+ try {
+ client.dropCatalog(OTHER_CATALOG);
+ } catch (NoSuchObjectException e) {
+ // NOP
+ }
+
+ // Clean up trash
+ metaStore.cleanWarehouseDirs();
+
+ new DatabaseBuilder().setName(OTHER_DATABASE).create(client, metaStore.getConf());
+
+ Catalog cat = new CatalogBuilder()
+ .setName(OTHER_CATALOG)
+ .setLocation(MetaStoreTestUtils.getTestWarehouseDir(OTHER_CATALOG))
+ .build();
+ client.createCatalog(cat);
+
+ // For this one don't specify a location to make sure it gets put in the catalog directory
+ inOtherCatalog = new DatabaseBuilder()
+ .setName(DATABASE_IN_OTHER_CATALOG)
+ .setCatalogName(OTHER_CATALOG)
+ .create(client, metaStore.getConf());
+
+ testTables[0] =
+ new TableBuilder()
+ .setTableName("test_table_1")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[1] =
+ new TableBuilder()
+ .setDbName(OTHER_DATABASE)
+ .setTableName("test_table_2")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[2] =
+ new TableBuilder()
+ .inDb(inOtherCatalog)
+ .setTableName("test_table_3")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ // Reload tables from the MetaStore
+ for(int i=0; i < testTables.length; i++) {
+ testTables[i] = client.getTable(testTables[i].getCatName(), testTables[i].getDbName(),
+ testTables[i].getTableName());
+ }
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ try {
+ if (client != null) {
+ try {
+ client.close();
+ } catch (Exception e) {
+ // HIVE-19729: Shallow the exceptions based on the discussion in the Jira
+ }
+ }
+ } finally {
+ client = null;
+ }
+ }
+
+
+ @Test
+ public void NoConstraints() throws TException{
+ Table table = testTables[0];
+
+ // Make sure get on a table with no key returns empty list
+ PrimaryKeysRequest rqst =
+ new PrimaryKeysRequest(table.getDbName(), table.getTableName());
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(),table.getTableName());
+
+ request.setCatName(table.getCatName());
+ rqst.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+ Assert.assertTrue(fetched.getDefaultConstraints().isEmpty());
+ Assert.assertTrue(fetched.getNotNullConstraints().isEmpty());
+ Assert.assertTrue(fetched.getPrimaryKeys().isEmpty());
+ Assert.assertTrue(fetched.getUniqueConstraints().isEmpty());
+ }
+
Review comment:
Also add tests for
1) all non-empty constraints
2) few constraints are present
3) more than 1 constraints are present of a single type on one table.
4) having multiple tables and then fetching the constraints
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##########
@@ -11286,6 +11287,20 @@ private String getPrimaryKeyConstraintName(String catName, String dbName, String
return notNullConstraints;
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
+ throws MetaException {
+ debugLog("Get all table constraints for the table - " + catName + "."+ db_name+"."+tbl_name + " in class ObjectStore.java");
+ SQLAllTableConstraints sqlAllTableConstraints = new SQLAllTableConstraints();
Review comment:
Let's have a followup jira to have overall one db call to fetch all constraints instead of having one for each type of constraint.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487551175
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched = false;
Review comment:
Currently we don't any implementation where we are calling the default constructor. But in order to have a parameterised constructor we need need a default constructor also. As a result of which in future it might possible that some implementation use. the default constructor. So its better to move back to flag based verification.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r477106020
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,145 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
+public class TestGetAllTableConstraints extends MetaStoreClientTest {
+ private static final String OTHER_DATABASE = "test_constraints_other_database";
+ private static final String OTHER_CATALOG = "test_constraints_other_catalog";
+ private static final String DATABASE_IN_OTHER_CATALOG = "test_constraints_database_in_other_catalog";
+ private final AbstractMetaStoreService metaStore;
+ private IMetaStoreClient client;
+ private Table[] testTables = new Table[3];
+ private Database inOtherCatalog;
+
+ public TestGetAllTableConstraints(String name, AbstractMetaStoreService metaStore) throws Exception {
+ this.metaStore = metaStore;
+ }
+ @Before
+ public void setUp() throws Exception {
+ // Get new client
+ client = metaStore.getClient();
+
+ // Clean up the database
+ client.dropDatabase(OTHER_DATABASE, true, true, true);
+ // Drop every table in the default database
+ for(String tableName : client.getAllTables(DEFAULT_DATABASE_NAME)) {
+ client.dropTable(DEFAULT_DATABASE_NAME, tableName, true, true, true);
+ }
+
+ client.dropDatabase(OTHER_CATALOG, DATABASE_IN_OTHER_CATALOG, true, true, true);
+ try {
+ client.dropCatalog(OTHER_CATALOG);
+ } catch (NoSuchObjectException e) {
+ // NOP
+ }
+
+ // Clean up trash
+ metaStore.cleanWarehouseDirs();
+
+ new DatabaseBuilder().setName(OTHER_DATABASE).create(client, metaStore.getConf());
+
+ Catalog cat = new CatalogBuilder()
+ .setName(OTHER_CATALOG)
+ .setLocation(MetaStoreTestUtils.getTestWarehouseDir(OTHER_CATALOG))
+ .build();
+ client.createCatalog(cat);
+
+ // For this one don't specify a location to make sure it gets put in the catalog directory
+ inOtherCatalog = new DatabaseBuilder()
+ .setName(DATABASE_IN_OTHER_CATALOG)
+ .setCatalogName(OTHER_CATALOG)
+ .create(client, metaStore.getConf());
+
+ testTables[0] =
+ new TableBuilder()
+ .setTableName("test_table_1")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[1] =
+ new TableBuilder()
+ .setDbName(OTHER_DATABASE)
+ .setTableName("test_table_2")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[2] =
+ new TableBuilder()
+ .inDb(inOtherCatalog)
+ .setTableName("test_table_3")
+ .addCol("col1", "int")
+ .addCol("col2", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ // Reload tables from the MetaStore
+ for(int i=0; i < testTables.length; i++) {
+ testTables[i] = client.getTable(testTables[i].getCatName(), testTables[i].getDbName(),
+ testTables[i].getTableName());
+ }
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ try {
+ if (client != null) {
+ try {
+ client.close();
+ } catch (Exception e) {
+ // HIVE-19729: Shallow the exceptions based on the discussion in the Jira
+ }
+ }
+ } finally {
+ client = null;
+ }
+ }
+
+
+ @Test
+ public void NoConstraints() throws TException{
+ Table table = testTables[0];
+
+ // Make sure get on a table with no key returns empty list
+ PrimaryKeysRequest rqst =
+ new PrimaryKeysRequest(table.getDbName(), table.getTableName());
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(),table.getTableName());
+
+ request.setCatName(table.getCatName());
+ rqst.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+ Assert.assertTrue(fetched.getDefaultConstraints().isEmpty());
+ Assert.assertTrue(fetched.getNotNullConstraints().isEmpty());
+ Assert.assertTrue(fetched.getPrimaryKeys().isEmpty());
+ Assert.assertTrue(fetched.getUniqueConstraints().isEmpty());
+ }
+
Review comment:
Test added
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375443
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487367583
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487367486
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r495126584
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -2502,6 +2520,9 @@ PartitionsResponse get_partitions_req(1:PartitionsRequest req)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
CheckConstraintsResponse get_check_constraints(1:CheckConstraintsRequest request)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
+ // All table constrains
+ AllTableConstraintsResponse get_all_table_constraints(1:AllTableConstraintsRequest request)
Review comment:
I have add the NoSuchObjectException in get_all_table_constraints.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r495126727
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9330,6 +9330,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all constraints attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
Review comment:
Added NoSuchObjectException
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r479451533
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
I think, it is redundant as HMS also logs the time taken by get_all_table_constraints api.
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Follow uniform naming style for arguments.
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Nit: Annotation and api signature can be in separate lines.
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
HMS api handles this default catalog name flow. Shall remove it here.
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,382 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.SQLCheckConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.List;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
+public class TestGetAllTableConstraints
+ extends MetaStoreClientTest {
+ private static final String OTHER_DATABASE = "test_constraints_other_database";
+ private static final String OTHER_CATALOG = "test_constraints_other_catalog";
+ private static final String DATABASE_IN_OTHER_CATALOG = "test_constraints_database_in_other_catalog";
+ private final AbstractMetaStoreService metaStore;
+ private IMetaStoreClient client;
+ private Table[] testTables = new Table[3];
+ private Database inOtherCatalog;
+
+ public TestGetAllTableConstraints(String name, AbstractMetaStoreService metaStore) throws Exception {
+ this.metaStore = metaStore;
+ }
+
+ @Before
+ public void setUp() throws Exception {
+ // Get new client
+ client = metaStore.getClient();
+
+ // Clean up the database
+ client.dropDatabase(OTHER_DATABASE, true, true, true);
+ // Drop every table in the default database
+ for (String tableName : client.getAllTables(DEFAULT_DATABASE_NAME)) {
+ client.dropTable(DEFAULT_DATABASE_NAME, tableName, true, true, true);
+ }
+
+ client.dropDatabase(OTHER_CATALOG, DATABASE_IN_OTHER_CATALOG, true, true, true);
+ try {
+ client.dropCatalog(OTHER_CATALOG);
+ } catch (NoSuchObjectException e) {
+ // NOP
+ }
+
+ // Clean up trash
+ metaStore.cleanWarehouseDirs();
+
+ new DatabaseBuilder().setName(OTHER_DATABASE).create(client, metaStore.getConf());
+
+ Catalog cat =
+ new CatalogBuilder().setName(OTHER_CATALOG).setLocation(MetaStoreTestUtils.getTestWarehouseDir(OTHER_CATALOG))
+ .build();
+ client.createCatalog(cat);
+
+ // For this one don't specify a location to make sure it gets put in the catalog directory
+ inOtherCatalog = new DatabaseBuilder().setName(DATABASE_IN_OTHER_CATALOG).setCatalogName(OTHER_CATALOG)
+ .create(client, metaStore.getConf());
+
+ testTables[0] = new TableBuilder().setTableName("test_table_1").addCol("col1", "int").addCol("col2", "int")
+ .addCol("col3", "boolean").addCol("col4", "int").addCol("col5", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[1] = new TableBuilder().setDbName(OTHER_DATABASE).setTableName("test_table_2").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ testTables[2] = new TableBuilder().inDb(inOtherCatalog).setTableName("test_table_3").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ // Reload tables from the MetaStore
+ for (int i = 0; i < testTables.length; i++) {
+ testTables[i] =
+ client.getTable(testTables[i].getCatName(), testTables[i].getDbName(), testTables[i].getTableName());
+ }
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ try {
+ if (client != null) {
+ try {
+ client.close();
+ } catch (Exception e) {
+ // HIVE-19729: Shallow the exceptions based on the discussion in the Jira
+ }
+ }
+ } finally {
+ client = null;
+ }
+ }
+
+ @Test
+ public void noConstraints() throws TException {
+ Table table = testTables[0];
+
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+ Assert.assertTrue(fetched.getDefaultConstraints().isEmpty());
+ Assert.assertTrue(fetched.getNotNullConstraints().isEmpty());
+ Assert.assertTrue(fetched.getPrimaryKeys().isEmpty());
+ Assert.assertTrue(fetched.getUniqueConstraints().isEmpty());
+ }
+
+ @Test
+ public void fewPresentWithMultipleConstraints() throws TException {
+ Table table = testTables[0];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col2 with not null constraint in default catalog and database;
+ String nnCol2ConstraintName = "col2_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(nnCol2ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol3 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol3);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
+ Assert.assertEquals(1, fetched.getPrimaryKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getPrimaryKeys().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getPrimaryKeys().get(0).getTable_name());
+ Assert.assertEquals("col1", fetched.getPrimaryKeys().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getPrimaryKeys().get(0).getKey_seq());
+ Assert.assertEquals(pkConstraintName, fetched.getPrimaryKeys().get(0).getPk_name());
+ Assert.assertTrue(fetched.getPrimaryKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getPrimaryKeys().get(0).getCatName());
+
+ // Assert unique constraint
+ Assert.assertEquals(1, fetched.getUniqueConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getUniqueConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getUniqueConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getUniqueConstraints().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getUniqueConstraints().get(0).getKey_seq());
+ Assert.assertEquals(uniqueConstraintName, fetched.getUniqueConstraints().get(0).getUk_name());
+ Assert.assertTrue(fetched.getUniqueConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getUniqueConstraints().get(0).getCatName());
+
+ // Assert Default constraint
+ Assert.assertEquals(1, fetched.getDefaultConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getDefaultConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getDefaultConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getDefaultConstraints().get(0).getColumn_name());
+ Assert.assertEquals("false", fetched.getDefaultConstraints().get(0).getDefault_value());
+ Assert.assertEquals(defaultConstraintName, fetched.getDefaultConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getDefaultConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getDefaultConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint
+ Assert.assertEquals(2, fetched.getNotNullConstraintsSize());
+
+ // Assert Not Null constraint for col2
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getNotNullConstraints().get(0).getColumn_name());
+ Assert.assertEquals(nnCol2ConstraintName, fetched.getNotNullConstraints().get(0).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint for col3
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(1).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(1).getTable_name());
+ Assert.assertEquals("col3", fetched.getNotNullConstraints().get(1).getColumn_name());
+ Assert.assertEquals(nnCol3ConstraintName, fetched.getNotNullConstraints().get(1).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(1).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(1).getCatName());
+
+ // Check constraints which is not present in table
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+
+ }
+
+ @Test
+ public void allConstraintsPresent() throws TException {
+ Table table = testTables[0];
+ Table parentTable = testTables[1];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col2 with not check constraint in default catalog and database;
+ String ccCol2ConstraintName = "col2_check";
+ List<SQLCheckConstraint> cc =
+ new SQLCheckConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(ccCol2ConstraintName)
+ .setCheckExpression("= 5").build(metaStore.getConf());
+ client.addCheckConstraint(cc);
+
+ // Set col1 of parent table to PK and Set Col4 of table to FK
+ String parentPkConstraintName = "parentpk";
+ List<SQLPrimaryKey> parentPk =
+ new SQLPrimaryKeyBuilder().onTable(parentTable).addColumn("col1").setConstraintName(parentPkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(parentPk);
+ String fkConstraintName = "fk";
+ List<SQLForeignKey> fk =
+ new SQLForeignKeyBuilder().fromPrimaryKey(parentPk).onTable(table).setConstraintName(fkConstraintName)
+ .addColumn("col4").build(metaStore.getConf());
+ client.addForeignKey(fk);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
+ Assert.assertEquals(1, fetched.getPrimaryKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getPrimaryKeys().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getPrimaryKeys().get(0).getTable_name());
+ Assert.assertEquals("col1", fetched.getPrimaryKeys().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getPrimaryKeys().get(0).getKey_seq());
+ Assert.assertEquals(pkConstraintName, fetched.getPrimaryKeys().get(0).getPk_name());
+ Assert.assertTrue(fetched.getPrimaryKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getPrimaryKeys().get(0).getCatName());
+
+ // Assert unique constraint
+ Assert.assertEquals(1, fetched.getUniqueConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getUniqueConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getUniqueConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getUniqueConstraints().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getUniqueConstraints().get(0).getKey_seq());
+ Assert.assertEquals(uniqueConstraintName, fetched.getUniqueConstraints().get(0).getUk_name());
+ Assert.assertTrue(fetched.getUniqueConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getUniqueConstraints().get(0).getCatName());
+
+ // Assert Default constraint
+ Assert.assertEquals(1, fetched.getDefaultConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getDefaultConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getDefaultConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getDefaultConstraints().get(0).getColumn_name());
+ Assert.assertEquals("false", fetched.getDefaultConstraints().get(0).getDefault_value());
+ Assert.assertEquals(defaultConstraintName, fetched.getDefaultConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getDefaultConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getDefaultConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint
+ Assert.assertEquals(1, fetched.getNotNullConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getNotNullConstraints().get(0).getColumn_name());
+ Assert.assertEquals(nnCol3ConstraintName, fetched.getNotNullConstraints().get(0).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(0).getCatName());
+
+ // Assert check constraint
+ Assert.assertEquals(1, fetched.getNotNullConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getCheckConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getCheckConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getCheckConstraints().get(0).getColumn_name());
+ Assert.assertEquals("= 5", fetched.getCheckConstraints().get(0).getCheck_expression());
+ Assert.assertEquals(ccCol2ConstraintName, fetched.getCheckConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getCheckConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getCheckConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getCheckConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getCheckConstraints().get(0).getCatName());
+
+ // Assert foreign key
+ Assert.assertEquals(1, fetched.getForeignKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getForeignKeys().get(0).getFktable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getForeignKeys().get(0).getFktable_name());
+ Assert.assertEquals("col4", fetched.getForeignKeys().get(0).getFkcolumn_name());
+ Assert.assertEquals(parentTable.getDbName(), fetched.getForeignKeys().get(0).getPktable_db());
+ Assert.assertEquals(parentTable.getTableName(), fetched.getForeignKeys().get(0).getPktable_name());
+ Assert.assertEquals("col1", fetched.getForeignKeys().get(0).getPkcolumn_name());
+ Assert.assertEquals(1, fetched.getForeignKeys().get(0).getKey_seq());
+ Assert.assertEquals(parentPkConstraintName, fetched.getForeignKeys().get(0).getPk_name());
+ Assert.assertEquals(fkConstraintName, fetched.getForeignKeys().get(0).getFk_name());
+ Assert.assertTrue(fetched.getForeignKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getForeignKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getForeignKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getForeignKeys().get(0).getCatName());
+
+ }
Review comment:
Add a negative scenario where db invalid or table invalid etc and check if we get NoSuchObjectException.
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,382 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.SQLCheckConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.List;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
Review comment:
Will these tests run in our PR build pipeline?
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
Nit: Naming can be dbName, tblName and catName.
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
Review comment:
This can be kept under isDebugEnabled flag.
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
Review comment:
Use this api in query planner instead of multiple calls.
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
Remove comma "," in last column.
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetAllTableConstraints.java
##########
@@ -0,0 +1,382 @@
+package org.apache.hadoop.hive.metastore.client;
+
+import org.apache.hadoop.hive.metastore.IMetaStoreClient;
+import org.apache.hadoop.hive.metastore.MetaStoreTestUtils;
+import org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest;
+import org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest;
+import org.apache.hadoop.hive.metastore.api.Catalog;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrimaryKeysRequest;
+import org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints;
+import org.apache.hadoop.hive.metastore.api.SQLCheckConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.client.builder.CatalogBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.DatabaseBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder;
+import org.apache.hadoop.hive.metastore.client.builder.TableBuilder;
+import org.apache.hadoop.hive.metastore.minihms.AbstractMetaStoreService;
+import org.apache.thrift.TException;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.List;
+
+import static org.apache.hadoop.hive.metastore.Warehouse.DEFAULT_DATABASE_NAME;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreCheckinTest.class)
+public class TestGetAllTableConstraints
+ extends MetaStoreClientTest {
+ private static final String OTHER_DATABASE = "test_constraints_other_database";
+ private static final String OTHER_CATALOG = "test_constraints_other_catalog";
+ private static final String DATABASE_IN_OTHER_CATALOG = "test_constraints_database_in_other_catalog";
+ private final AbstractMetaStoreService metaStore;
+ private IMetaStoreClient client;
+ private Table[] testTables = new Table[3];
+ private Database inOtherCatalog;
+
+ public TestGetAllTableConstraints(String name, AbstractMetaStoreService metaStore) throws Exception {
+ this.metaStore = metaStore;
+ }
+
+ @Before
+ public void setUp() throws Exception {
+ // Get new client
+ client = metaStore.getClient();
+
+ // Clean up the database
+ client.dropDatabase(OTHER_DATABASE, true, true, true);
+ // Drop every table in the default database
+ for (String tableName : client.getAllTables(DEFAULT_DATABASE_NAME)) {
+ client.dropTable(DEFAULT_DATABASE_NAME, tableName, true, true, true);
+ }
+
+ client.dropDatabase(OTHER_CATALOG, DATABASE_IN_OTHER_CATALOG, true, true, true);
+ try {
+ client.dropCatalog(OTHER_CATALOG);
+ } catch (NoSuchObjectException e) {
+ // NOP
+ }
+
+ // Clean up trash
+ metaStore.cleanWarehouseDirs();
+
+ new DatabaseBuilder().setName(OTHER_DATABASE).create(client, metaStore.getConf());
+
+ Catalog cat =
+ new CatalogBuilder().setName(OTHER_CATALOG).setLocation(MetaStoreTestUtils.getTestWarehouseDir(OTHER_CATALOG))
+ .build();
+ client.createCatalog(cat);
+
+ // For this one don't specify a location to make sure it gets put in the catalog directory
+ inOtherCatalog = new DatabaseBuilder().setName(DATABASE_IN_OTHER_CATALOG).setCatalogName(OTHER_CATALOG)
+ .create(client, metaStore.getConf());
+
+ testTables[0] = new TableBuilder().setTableName("test_table_1").addCol("col1", "int").addCol("col2", "int")
+ .addCol("col3", "boolean").addCol("col4", "int").addCol("col5", "varchar(32)")
+ .create(client, metaStore.getConf());
+
+ testTables[1] = new TableBuilder().setDbName(OTHER_DATABASE).setTableName("test_table_2").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ testTables[2] = new TableBuilder().inDb(inOtherCatalog).setTableName("test_table_3").addCol("col1", "int")
+ .addCol("col2", "varchar(32)").create(client, metaStore.getConf());
+
+ // Reload tables from the MetaStore
+ for (int i = 0; i < testTables.length; i++) {
+ testTables[i] =
+ client.getTable(testTables[i].getCatName(), testTables[i].getDbName(), testTables[i].getTableName());
+ }
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ try {
+ if (client != null) {
+ try {
+ client.close();
+ } catch (Exception e) {
+ // HIVE-19729: Shallow the exceptions based on the discussion in the Jira
+ }
+ }
+ } finally {
+ client = null;
+ }
+ }
+
+ @Test
+ public void noConstraints() throws TException {
+ Table table = testTables[0];
+
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+ Assert.assertTrue(fetched.getDefaultConstraints().isEmpty());
+ Assert.assertTrue(fetched.getNotNullConstraints().isEmpty());
+ Assert.assertTrue(fetched.getPrimaryKeys().isEmpty());
+ Assert.assertTrue(fetched.getUniqueConstraints().isEmpty());
+ }
+
+ @Test
+ public void fewPresentWithMultipleConstraints() throws TException {
+ Table table = testTables[0];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col2 with not null constraint in default catalog and database;
+ String nnCol2ConstraintName = "col2_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(nnCol2ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol3 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol3);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
+ Assert.assertEquals(1, fetched.getPrimaryKeysSize());
+ Assert.assertEquals(table.getDbName(), fetched.getPrimaryKeys().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getPrimaryKeys().get(0).getTable_name());
+ Assert.assertEquals("col1", fetched.getPrimaryKeys().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getPrimaryKeys().get(0).getKey_seq());
+ Assert.assertEquals(pkConstraintName, fetched.getPrimaryKeys().get(0).getPk_name());
+ Assert.assertTrue(fetched.getPrimaryKeys().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getPrimaryKeys().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getPrimaryKeys().get(0).getCatName());
+
+ // Assert unique constraint
+ Assert.assertEquals(1, fetched.getUniqueConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getUniqueConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getUniqueConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getUniqueConstraints().get(0).getColumn_name());
+ Assert.assertEquals(1, fetched.getUniqueConstraints().get(0).getKey_seq());
+ Assert.assertEquals(uniqueConstraintName, fetched.getUniqueConstraints().get(0).getUk_name());
+ Assert.assertTrue(fetched.getUniqueConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getUniqueConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getUniqueConstraints().get(0).getCatName());
+
+ // Assert Default constraint
+ Assert.assertEquals(1, fetched.getDefaultConstraintsSize());
+ Assert.assertEquals(table.getDbName(), fetched.getDefaultConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getDefaultConstraints().get(0).getTable_name());
+ Assert.assertEquals("col3", fetched.getDefaultConstraints().get(0).getColumn_name());
+ Assert.assertEquals("false", fetched.getDefaultConstraints().get(0).getDefault_value());
+ Assert.assertEquals(defaultConstraintName, fetched.getDefaultConstraints().get(0).getDc_name());
+ Assert.assertTrue(fetched.getDefaultConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getDefaultConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getDefaultConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint
+ Assert.assertEquals(2, fetched.getNotNullConstraintsSize());
+
+ // Assert Not Null constraint for col2
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(0).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(0).getTable_name());
+ Assert.assertEquals("col2", fetched.getNotNullConstraints().get(0).getColumn_name());
+ Assert.assertEquals(nnCol2ConstraintName, fetched.getNotNullConstraints().get(0).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(0).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(0).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(0).getCatName());
+
+ // Assert Not Null constraint for col3
+ Assert.assertEquals(table.getDbName(), fetched.getNotNullConstraints().get(1).getTable_db());
+ Assert.assertEquals(table.getTableName(), fetched.getNotNullConstraints().get(1).getTable_name());
+ Assert.assertEquals("col3", fetched.getNotNullConstraints().get(1).getColumn_name());
+ Assert.assertEquals(nnCol3ConstraintName, fetched.getNotNullConstraints().get(1).getNn_name());
+ Assert.assertTrue(fetched.getNotNullConstraints().get(1).isEnable_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isValidate_cstr());
+ Assert.assertFalse(fetched.getNotNullConstraints().get(1).isRely_cstr());
+ Assert.assertEquals(table.getCatName(), fetched.getNotNullConstraints().get(1).getCatName());
+
+ // Check constraints which is not present in table
+ Assert.assertTrue(fetched.getCheckConstraints().isEmpty());
+ Assert.assertTrue(fetched.getForeignKeys().isEmpty());
+
+ }
+
+ @Test
+ public void allConstraintsPresent() throws TException {
+ Table table = testTables[0];
+ Table parentTable = testTables[1];
+
+ // Set col1 as primary key Constraint in default catalog and database
+ String pkConstraintName = "col1_pk";
+ List<SQLPrimaryKey> pk =
+ new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName(pkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(pk);
+
+ // Set col2 with Unique Constraint in default catalog and database
+ String uniqueConstraintName = "col2_unique";
+ List<SQLUniqueConstraint> uc =
+ new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(uniqueConstraintName)
+ .build(metaStore.getConf());
+ client.addUniqueConstraint(uc);
+
+ // Set col3 with default Constraint in default catalog and database
+ String defaultConstraintName = "col3_default";
+ List<SQLDefaultConstraint> dv =
+ new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(defaultConstraintName)
+ .setDefaultVal(false).build(metaStore.getConf());
+ client.addDefaultConstraint(dv);
+
+ // Set col3 with not null constraint in default catalog and database;
+ String nnCol3ConstraintName = "col3_not_null";
+ List<SQLNotNullConstraint> nnCol2 =
+ new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName(nnCol3ConstraintName)
+ .build(metaStore.getConf());
+ client.addNotNullConstraint(nnCol2);
+
+ // Set col2 with not check constraint in default catalog and database;
+ String ccCol2ConstraintName = "col2_check";
+ List<SQLCheckConstraint> cc =
+ new SQLCheckConstraintBuilder().onTable(table).addColumn("col2").setConstraintName(ccCol2ConstraintName)
+ .setCheckExpression("= 5").build(metaStore.getConf());
+ client.addCheckConstraint(cc);
+
+ // Set col1 of parent table to PK and Set Col4 of table to FK
+ String parentPkConstraintName = "parentpk";
+ List<SQLPrimaryKey> parentPk =
+ new SQLPrimaryKeyBuilder().onTable(parentTable).addColumn("col1").setConstraintName(parentPkConstraintName)
+ .build(metaStore.getConf());
+ client.addPrimaryKey(parentPk);
+ String fkConstraintName = "fk";
+ List<SQLForeignKey> fk =
+ new SQLForeignKeyBuilder().fromPrimaryKey(parentPk).onTable(table).setConstraintName(fkConstraintName)
+ .addColumn("col4").build(metaStore.getConf());
+ client.addForeignKey(fk);
+
+ // Fetch all constraints for the table in default catalog and database
+ AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName());
+ request.setCatName(table.getCatName());
+ SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
+
+ // Assert primary key constraint
Review comment:
Too much of duplicate code. Can write common methods for the asserts and use it in all tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487538671
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5664,69 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
- }
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName)
+ throws HiveException, NoSuchObjectException {
try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
+ return getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName, getDefaultCatalog(conf)));
} catch (NoSuchObjectException e) {
Review comment:
Will NoSuchObjectException comes here? From the interface, it sounds not.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
ok
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r495170058
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getTableConstraints(this.getDbName(), this.getTableName(), true, true);
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
Review comment:
If exception generated then fall back to default constructor. As all the constraint will set to null which was the previous contract.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375471
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487366456
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487375692
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sankarh commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
sankarh commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r486978810
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Can add utility method to check for null and empty of given list. Used multiple times. Also use local variables to reduce the code.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
nit: Use "fetchReliable" and "fetchEnabled" instead of "reliable" and "enable" as it sound like flag to enable something.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
nit: Add space before and after "=".
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
Method won't return NoSuchObjectException. Shall remove it here and from all implementations.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
getReliableAndEnableTableConstraints() method name is confusing. It is used in only one place and hence, just expose getTableConstraints() itself.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
nit: Add space after comma between arguments.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
Log dbName.tblName.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Other constraints also needs to be filtered based on "enable" flag, right?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
nit: Add space after comma.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Just add a note that we don't make copy of constraints we just refer to same input constraints object.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
Why is it static method? In both the callers, we pass tbl.getTableConstraintsInfo(). This can be a public method without arguments inside "Table" class.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Do we need an extra flag? Can't we just check null for tableConstraintsInfo?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Shall make all members private as we have get and set methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashish-kumar-sharma commented on a change in pull request #1419: HIVE-22782: Consolidate metastore call to fetch constraints
Posted by GitBox <gi...@apache.org>.
ashish-kumar-sharma commented on a change in pull request #1419:
URL: https://github.com/apache/hive/pull/1419#discussion_r487366456
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
Review comment:
Done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##########
@@ -1155,22 +1150,19 @@ void dumpConstraintMetadata(String dbName, String tblName, Path dbRoot, Hive hiv
Path constraintsRoot = new Path(dbRoot, ReplUtils.CONSTRAINTS_ROOT_DIR_NAME);
Path commonConstraintsFile = new Path(constraintsRoot, ConstraintFileType.COMMON.getPrefix() + tblName);
Path fkConstraintsFile = new Path(constraintsRoot, ConstraintFileType.FOREIGNKEY.getPrefix() + tblName);
- List<SQLPrimaryKey> pks = hiveDb.getPrimaryKeyList(dbName, tblName);
- List<SQLForeignKey> fks = hiveDb.getForeignKeyList(dbName, tblName);
- List<SQLUniqueConstraint> uks = hiveDb.getUniqueConstraintList(dbName, tblName);
- List<SQLNotNullConstraint> nns = hiveDb.getNotNullConstraintList(dbName, tblName);
- if ((pks != null && !pks.isEmpty()) || (uks != null && !uks.isEmpty())
- || (nns != null && !nns.isEmpty())) {
+ SQLAllTableConstraints tableConstraints = hiveDb.getTableConstraints(dbName,tblName);
+ if ((tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) || (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty())
Review comment:
Yes, code is very redundant. I have replaced it with CollectionsUtils.isNotEmpty() which does the same check i.e not null and isEmpty()
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
Review comment:
Update the variable name.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, true, true);
}
- /**
- * Get primary key columns associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getReliablePrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, true);
- }
-
- private PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName, boolean onlyReliable)
+ private TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean reliable, boolean enable)
throws HiveException {
PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_PK);
- try {
- List<SQLPrimaryKey> primaryKeys = getMSC().getPrimaryKeys(new PrimaryKeysRequest(dbName, tblName));
- if (onlyReliable && primaryKeys != null && !primaryKeys.isEmpty()) {
- primaryKeys = primaryKeys.stream()
- .filter(pk -> pk.isRely_cstr())
- .collect(Collectors.toList());
- }
-
- return new PrimaryKeyInfo(primaryKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_PK, "HS2-cache");
- }
- }
-
- /**
- * Get all foreign keys associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, false);
- }
-
- /**
- * Get foreign keys associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Foreign keys associated with the table.
- * @throws HiveException
- */
- public ForeignKeyInfo getReliableForeignKeys(String dbName, String tblName) throws HiveException {
- return getForeignKeys(dbName, tblName, true);
- }
-
- private ForeignKeyInfo getForeignKeys(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_FK);
- try {
- List<SQLForeignKey> foreignKeys = getMSC().getForeignKeys(new ForeignKeysRequest(null, null, dbName, tblName));
- if (onlyReliable && foreignKeys != null && !foreignKeys.isEmpty()) {
- foreignKeys = foreignKeys.stream()
- .filter(fk -> fk.isRely_cstr())
- .collect(Collectors.toList());
+ perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
+ try {
+ SQLAllTableConstraints tableConstraints =
+ getMSC().getAllTableConstraints(new AllTableConstraintsRequest(dbName, tblName));
+ if (reliable && tableConstraints != null) {
+ if (tableConstraints.getPrimaryKeys() != null && !tableConstraints.getPrimaryKeys().isEmpty()) {
+ tableConstraints.setPrimaryKeys(
+ tableConstraints.getPrimaryKeys().stream().filter(primaryKey -> primaryKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getForeignKeys() != null && !tableConstraints.getForeignKeys().isEmpty()) {
+ tableConstraints.setForeignKeys(
+ tableConstraints.getForeignKeys().stream().filter(foreignKey -> foreignKey.isRely_cstr())
+ .collect(Collectors.toList()));
+ }
+ if (tableConstraints.getUniqueConstraints() != null && !tableConstraints.getUniqueConstraints().isEmpty()) {
+ tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream()
+ .filter(uniqueConstraint -> uniqueConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
+ if (tableConstraints.getNotNullConstraints() != null && !tableConstraints.getNotNullConstraints().isEmpty()) {
+ tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream()
+ .filter(notNullConstraint -> notNullConstraint.isRely_cstr()).collect(Collectors.toList()));
+ }
}
- return new ForeignKeyInfo(foreignKeys, tblName, dbName);
- } catch (Exception e) {
- throw new HiveException(e);
- } finally {
- perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_FK, "HS2-cache");
- }
- }
-
- /**
- * Get all unique constraints associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, false);
- }
-
- /**
- * Get unique constraints associated with the table that are available for optimization.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Unique constraints associated with the table.
- * @throws HiveException
- */
- public UniqueConstraint getReliableUniqueConstraints(String dbName, String tblName) throws HiveException {
- return getUniqueConstraints(dbName, tblName, true);
- }
-
- private UniqueConstraint getUniqueConstraints(String dbName, String tblName, boolean onlyReliable)
- throws HiveException {
- PerfLogger perfLogger = SessionState.getPerfLogger();
- perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_UNIQ_CONSTRAINT);
- try {
- List<SQLUniqueConstraint> uniqueConstraints = getMSC().getUniqueConstraints(
- new UniqueConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
- if (onlyReliable && uniqueConstraints != null && !uniqueConstraints.isEmpty()) {
- uniqueConstraints = uniqueConstraints.stream()
- .filter(uk -> uk.isRely_cstr())
- .collect(Collectors.toList());
+ if (enable && tableConstraints != null) {
Review comment:
Yes, Ideally we can fetch all constraint on rely,enable and validate. But in table.java we have chooses to pk,fk,unique and notnull as fetch based on rely. Default and check constraints are fetch based on enable. this method is tightly couple with Hive.java class that's why i have put everything is one method as tableConstraints is single object. But yes we can have separate function for both of them
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -5661,184 +5663,79 @@ public void dropConstraint(String dbName, String tableName, String constraintNam
}
}
- public List<SQLDefaultConstraint> getDefaultConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
+ public SQLAllTableConstraints getTableConstraints(String dbName, String tblName) throws HiveException, NoSuchObjectException {
try {
- return getMSC().getDefaultConstraints(new DefaultConstraintsRequest(getDefaultCatalog(conf), dbName, tblName));
+ AllTableConstraintsRequest tableConstraintsRequest = new AllTableConstraintsRequest();
+ tableConstraintsRequest.setDbName(dbName);
+ tableConstraintsRequest.setTblName(tblName);
+ tableConstraintsRequest.setCatName(getDefaultCatalog(conf));
+ return getMSC().getAllTableConstraints(tableConstraintsRequest);
} catch (NoSuchObjectException e) {
throw e;
} catch (Exception e) {
throw new HiveException(e);
}
}
-
- public List<SQLCheckConstraint> getCheckConstraintList(String dbName, String tblName) throws HiveException, NoSuchObjectException {
- try {
- return getMSC().getCheckConstraints(new CheckConstraintsRequest(getDefaultCatalog(conf),
- dbName, tblName));
- } catch (NoSuchObjectException e) {
- throw e;
- } catch (Exception e) {
- throw new HiveException(e);
- }
+ public TableConstraintsInfo getAllTableConstraints(String dbName, String tblName) throws HiveException {
+ return getTableConstraints(dbName, tblName, false, false);
}
- /**
- * Get all primary key columns associated with the table.
- *
- * @param dbName Database Name
- * @param tblName Table Name
- * @return Primary Key associated with the table.
- * @throws HiveException
- */
- public PrimaryKeyInfo getPrimaryKeys(String dbName, String tblName) throws HiveException {
- return getPrimaryKeys(dbName, tblName, false);
+ public TableConstraintsInfo getReliableAndEnableTableConstraints(String dbName, String tblName) throws HiveException {
Review comment:
Agreed, Removed this wrapper function
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
##########
@@ -1014,6 +1015,11 @@ public FileMetadataHandler getFileMetadataHandler(FileMetadataExprType type) {
return null;
}
+ @Override public SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
##########
@@ -1485,6 +1485,16 @@ void getFileMetadataByExpr(List<Long> fileIds, FileMetadataExprType type, byte[]
List<SQLCheckConstraint> getCheckConstraints(String catName, String db_name,
String tbl_name) throws MetaException;
+ /**
+ * Get all constraints of the table
+ * @param catName catalog name
+ * @param db_name database name
+ * @param tbl_name table name
+ * @return all constraints for this table
+ * @throws MetaException error accessing the RDBMS
+ */
+ SQLAllTableConstraints getAllTableConstraints(String catName, String db_name, String tbl_name)
Review comment:
Done
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
+ req.setCatName(getDefaultCatalog(conf));
+ }
+
+ return client.get_all_table_constraints(req).getAllTableConstraints();
+ } finally {
+ long diff = System.currentTimeMillis() - t1;
Review comment:
done
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -720,6 +729,15 @@ struct CheckConstraintsResponse {
1: required list<SQLCheckConstraint> checkConstraints
}
+struct AllTableConstraintsRequest {
+ 1: required string db_name,
Review comment:
set all variable to camel casing
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -2811,6 +2811,26 @@ public GetFieldsResponse getFieldsRequest(GetFieldsRequest req)
return client.get_check_constraints(req).getCheckConstraints();
}
+ @Override
+ public SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest req)
+ throws MetaException, NoSuchObjectException, TException {
+ long t1 = System.currentTimeMillis();
+
+ try {
+ if (!req.isSetCatName()) {
Review comment:
removed
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -122,6 +122,15 @@ struct SQLCheckConstraint {
9: bool rely_cstr // Rely/No Rely
}
+struct SQLAllTableConstraints {
+ 1: optional list<SQLPrimaryKey> primaryKeys,
+ 2: optional list<SQLForeignKey> foreignKeys,
+ 3: optional list<SQLUniqueConstraint> uniqueConstraints,
+ 4: optional list<SQLNotNullConstraint> notNullConstraints,
+ 5: optional list<SQLDefaultConstraint> defaultConstraints,
+ 6: optional list<SQLCheckConstraint> checkConstraints,
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/TableConstraintsInfo.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.metadata;
+
+public class TableConstraintsInfo {
+ PrimaryKeyInfo primaryKeyInfo;
+ ForeignKeyInfo foreignKeyInfo;
+ UniqueConstraint uniqueConstraint;
+ DefaultConstraint defaultConstraint;
+ CheckConstraint checkConstraint;
+ NotNullConstraint notNullConstraint;
+
+ public TableConstraintsInfo() {
+ }
+
+ public TableConstraintsInfo(PrimaryKeyInfo primaryKeyInfo, ForeignKeyInfo foreignKeyInfo,
+ UniqueConstraint uniqueConstraint, DefaultConstraint defaultConstraint, CheckConstraint checkConstraint,
+ NotNullConstraint notNullConstraint) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ this.foreignKeyInfo = foreignKeyInfo;
+ this.uniqueConstraint = uniqueConstraint;
+ this.defaultConstraint = defaultConstraint;
+ this.checkConstraint = checkConstraint;
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ return primaryKeyInfo;
+ }
+
+ public void setPrimaryKeyInfo(PrimaryKeyInfo primaryKeyInfo) {
+ this.primaryKeyInfo = primaryKeyInfo;
+ }
+
+ public ForeignKeyInfo getForeignKeyInfo() {
+ return foreignKeyInfo;
+ }
+
+ public void setForeignKeyInfo(ForeignKeyInfo foreignKeyInfo) {
+ this.foreignKeyInfo = foreignKeyInfo;
+ }
+
+ public UniqueConstraint getUniqueConstraint() {
+ return uniqueConstraint;
+ }
+
+ public void setUniqueConstraint(UniqueConstraint uniqueConstraint) {
+ this.uniqueConstraint = uniqueConstraint;
+ }
+
+ public DefaultConstraint getDefaultConstraint() {
+ return defaultConstraint;
+ }
+
+ public void setDefaultConstraint(DefaultConstraint defaultConstraint) {
+ this.defaultConstraint = defaultConstraint;
+ }
+
+ public CheckConstraint getCheckConstraint() {
+ return checkConstraint;
+ }
+
+ public void setCheckConstraint(CheckConstraint checkConstraint) {
+ this.checkConstraint = checkConstraint;
+ }
+
+ public NotNullConstraint getNotNullConstraint() {
+ return notNullConstraint;
+ }
+
+ public void setNotNullConstraint(NotNullConstraint notNullConstraint) {
+ this.notNullConstraint = notNullConstraint;
+ }
+
+ public static boolean isTableConstraintsInfoNotEmpty(TableConstraintsInfo info) {
Review comment:
i was trying to maintain the code uniformity. But since we have a wrapper class in places we can remove all static method and follow OOD para-dime
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##########
@@ -9200,6 +9200,31 @@ public CheckConstraintsResponse get_check_constraints(CheckConstraintsRequest re
return new CheckConstraintsResponse(ret);
}
+ /**
+ * Api to fetch all table constraints at once
+ * @param request it consist of catalog name, database name and table name to identify the table in metastore
+ * @return all cnstraint attached to given table
+ * @throws TException
+ */
+ @Override
+ public AllTableConstraintsResponse get_all_table_constraints(AllTableConstraintsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
+ String dbName = request.getDbName();
+ String tblName = request.getTblName();
+ startTableFunction("get_all_table_constraints", catName, dbName, tblName);
+ SQLAllTableConstraints ret = null;
+ Exception ex = null;
+ try {
+ ret = getMS().getAllTableConstraints(catName,dbName,tblName);
Review comment:
fixed
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
##########
@@ -3631,6 +3631,17 @@ boolean cacheFileMetadata(String dbName, String tableName, String partName,
List<SQLCheckConstraint> getCheckConstraints(CheckConstraintsRequest request) throws MetaException,
NoSuchObjectException, TException;
+ /**
+ * Get all constraints of given table
+ * @param request Request info
+ * @return all constraints of this table
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ * @throws TException
+ */
+ SQLAllTableConstraints getAllTableConstraints(AllTableConstraintsRequest request)
Review comment:
removed
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
done
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
Review comment:
replaced with complete name
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -116,22 +116,12 @@
private transient Boolean outdatedForRewritingMaterializedView;
/** Constraint related objects */
- private transient PrimaryKeyInfo pki;
- private transient ForeignKeyInfo fki;
- private transient UniqueConstraint uki;
- private transient NotNullConstraint nnc;
- private transient DefaultConstraint dc;
- private transient CheckConstraint cc;
+ private transient TableConstraintsInfo tableConstraintsInfo;
/** Constraint related flags
* This is to track if constraints are retrieved from metastore or not
*/
- private transient boolean isPKFetched=false;
- private transient boolean isFKFetched=false;
- private transient boolean isUniqueFetched=false;
- private transient boolean isNotNullFetched=false;
- private transient boolean isDefaultFetched=false;
- private transient boolean isCheckFetched=false;
+ private transient boolean isTableConstraintsFetched=false;
Review comment:
Since we have wrapper in place. Now we don't require extra flag to track the object is fetched or not. We can have if condition to check wrapper is null or not
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##########
@@ -1176,145 +1166,101 @@ public void setStatsStateLikeNewTable() {
* Note that set apis are used by DESCRIBE only, although get apis return RELY or ENABLE
* constraints DESCRIBE could set all type of constraints
* */
-
- /* This only return PK which are created with RELY */
- public PrimaryKeyInfo getPrimaryKeyInfo() {
- if(!this.isPKFetched) {
+ public TableConstraintsInfo getTableConstraintsInfo() {
+ if (!this.isTableConstraintsFetched) {
try {
- pki = Hive.get().getReliablePrimaryKeys(this.getDbName(), this.getTableName());
- this.isPKFetched = true;
+ tableConstraintsInfo = Hive.get().getReliableAndEnableTableConstraints(this.getDbName(), this.getTableName());
+ this.isTableConstraintsFetched = true;
} catch (HiveException e) {
- LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
+ LOG.warn(
+ "Cannot retrieve table constraints info for table : " + this.getTableName() + " ignoring exception: " + e);
}
}
- return pki;
+ return tableConstraintsInfo;
}
- public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
- this.pki = pki;
- this.isPKFetched = true;
+ /**
+ * TableConstraintsInfo setter
+ * @param tableConstraintsInfo
+ */
+ public void setTableConstraintsInfo(TableConstraintsInfo tableConstraintsInfo) {
+ this.tableConstraintsInfo = tableConstraintsInfo;
+ this.isTableConstraintsFetched = true;
}
- /* This only return FK constraints which are created with RELY */
- public ForeignKeyInfo getForeignKeyInfo() {
- if(!isFKFetched) {
- try {
- fki = Hive.get().getReliableForeignKeys(this.getDbName(), this.getTableName());
- this.isFKFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve FK info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ /**
+ * This only return PK which are created with RELY
+ * @return primary key constraint list
+ */
+ public PrimaryKeyInfo getPrimaryKeyInfo() {
+ if (!this.isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return fki;
+ return tableConstraintsInfo.getPrimaryKeyInfo();
}
- public void setForeignKeyInfo(ForeignKeyInfo fki) {
- this.fki = fki;
- this.isFKFetched = true;
+ /**
+ * This only return FK constraints which are created with RELY
+ * @return foreign key constraint list
+ */
+ public ForeignKeyInfo getForeignKeyInfo() {
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
+ }
+ return tableConstraintsInfo.getForeignKeyInfo();
}
- /* This only return UNIQUE constraint defined with RELY */
+ /**
+ * This only return UNIQUE constraint defined with RELY
+ * @return unique constraint list
+ */
public UniqueConstraint getUniqueKeyInfo() {
- if(!isUniqueFetched) {
- try {
- uki = Hive.get().getReliableUniqueConstraints(this.getDbName(), this.getTableName());
- this.isUniqueFetched = true;
- } catch (HiveException e) {
- LOG.warn(
- "Cannot retrieve Unique Key info for table : " + this.getTableName()
- + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return uki;
- }
-
- public void setUniqueKeyInfo(UniqueConstraint uki) {
- this.uki = uki;
- this.isUniqueFetched = true;
+ return tableConstraintsInfo.getUniqueConstraint();
}
- /* This only return NOT NULL constraint defined with RELY */
+ /**
+ * This only return NOT NULL constraint defined with RELY
+ * @return not null constraint list
+ */
public NotNullConstraint getNotNullConstraint() {
- if(!isNotNullFetched) {
- try {
- nnc = Hive.get().getReliableNotNullConstraints(this.getDbName(), this.getTableName());
- this.isNotNullFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Not Null constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return nnc;
- }
-
- public void setNotNullConstraint(NotNullConstraint nnc) {
- this.nnc = nnc;
- this.isNotNullFetched = true;
+ return tableConstraintsInfo.getNotNullConstraint();
}
- /* This only return DEFAULT constraint defined with ENABLE */
+ /**
+ * This only return DEFAULT constraint defined with ENABLE
+ * @return default constraint list
+ */
public DefaultConstraint getDefaultConstraint() {
- if(!isDefaultFetched) {
- try {
- dc = Hive.get().getEnabledDefaultConstraints(this.getDbName(), this.getTableName());
- this.isDefaultFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Default constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return dc;
+ return tableConstraintsInfo.getDefaultConstraint();
}
- public void setDefaultConstraint(DefaultConstraint dc) {
- this.dc = dc;
- this.isDefaultFetched = true;
- }
-
- /* This only return CHECK constraint defined with ENABLE */
+ /**
+ * This only return CHECK constraint defined with ENABLE
+ * @return check constraint list
+ */
public CheckConstraint getCheckConstraint() {
- if(!isCheckFetched) {
- try{
- cc = Hive.get().getEnabledCheckConstraints(this.getDbName(), this.getTableName());
- this.isCheckFetched = true;
- } catch (HiveException e) {
- LOG.warn("Cannot retrieve Check constraint info for table : "
- + this.getTableName() + " ignoring exception: " + e);
- }
+ if (!isTableConstraintsFetched) {
+ getTableConstraintsInfo();
}
- return cc;
- }
-
- public void setCheckConstraint(CheckConstraint cc) {
- this.cc = cc;
- this.isCheckFetched = true;
+ return tableConstraintsInfo.getCheckConstraint();
}
/** This shouldn't use get apis because those api call metastore
* to fetch constraints.
* getMetaData only need to make a copy of existing constraints, even if those are not fetched
*/
public void copyConstraints(final Table tbl) {
- this.pki = tbl.pki;
- this.isPKFetched = tbl.isPKFetched;
-
- this.fki = tbl.fki;
- this.isFKFetched = tbl.isFKFetched;
-
- this.uki = tbl.uki;
- this.isUniqueFetched = tbl.isUniqueFetched;
-
- this.nnc = tbl.nnc;
- this.isNotNullFetched = tbl.isNotNullFetched;
-
- this.dc = tbl.dc;
- this.isDefaultFetched = tbl.isDefaultFetched;
-
- this.cc = tbl.cc;
- this.isCheckFetched = tbl.isCheckFetched;
+ this.tableConstraintsInfo = tbl.tableConstraintsInfo;
Review comment:
Since the legacy way is to just copy the reference. Shouldn't I create a deep copy instead of reference?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org