You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Miklos Gergely <mg...@hortonworks.com> on 2019/05/05 18:12:29 UTC
Review Request 70597: Break up DDLTask - extract Process related operations

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70597/
-----------------------------------------------------------

Review request for hive and Zoltan Haindrich.


Bugs: HIVE-21693
    https://issues.apache.org/jira/browse/HIVE-21693


Repository: hive-git


Description
-------

DDLTask is a huge class, more than 5000 lines long. The related DDLWork is also a huge class, which has a field for each DDL operation it supports. The goal is to refactor these in order to have everything cut into more handleable classes under the package  org.apache.hadoop.hive.ql.exec.ddl:

have a separate class for each operation
have a package for each operation group (database ddl, table ddl, etc), so the amount of classes under a package is more manageable
make all the requests (DDLDesc subclasses) immutable
DDLTask should be agnostic to the actual operations
right now let's ignore the issue of having some operations handled by DDLTask which are not actual DDL operations (lock, unlock, desc...)
In the interim time when there are two DDLTask and DDLWork classes in the code base the new ones in the new package are called DDLTask2 and DDLWork2 thus avoiding the usage of fully qualified class names where both the old and the new classes are in use.

Step #7: extract all the process related operations from the old DDLTask, and move them under the new package.

Also rearranged table and view related operations for a nicer structure.


Diffs
-----

  hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/CreateTableHook.java 8366e5ec93 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 6d9dd5eb05 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/metadata/DummySemanticAnalyzerHook.java f988d42366 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/metadata/DummySemanticAnalyzerHook1.java b2b00720c6 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/alter/AlterMaterializedViewDesc.java f0afccacb5 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/alter/AlterMaterializedViewRewriteDesc.java 72cc84c5cd 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/alter/AlterMaterializedViewRewriteOperation.java 379f472452 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/alter/package-info.java b113f39db8 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/AbortTransactionsDesc.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/AbortTransactionsOperation.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/KillQueriesOperation.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/ShowCompactionsOperation.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/ShowTransactionsDesc.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/ShowTransactionsOperation.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/process/package-info.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateTableDesc.java ee32f4c9b4 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateTableLikeDesc.java 6652b79a88 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateTableLikeOperation.java 7cec1e38af 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateTableOperation.java 2987cab9b0 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateViewDesc.java ebc7e00b7d 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateViewOperation.java f8d95aa747 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/DescTableDesc.java bb533c2c85 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/DescTableOperation.java 1d94ff3a5b 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/DropTableDesc.java f910c57f3a 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/DropTableOperation.java d250772b2a 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/LockTableDesc.java 2a8b02e067 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/LockTableOperation.java 2044a81406 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/PreInsertTableDesc.java 4bb609ef54 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/PreInsertTableOperation.java 5d85d0a14c 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowCreateTableDesc.java a06f1fae5f 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowCreateTableOperation.java 932d9428a9 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTablePropertiesDesc.java 7ba1c2daef 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTablePropertiesOperation.java 385052d705 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTableStatusDesc.java 6707350f34 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTableStatusOperation.java ea695fd1a3 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTablesDesc.java 9ec390483d 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTablesOperation.java 71b5717bb8 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/TruncateTableDesc.java 1f0cd82690 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/TruncateTableOperation.java 9778bfac92 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/UnlockTableDesc.java 86050244eb 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/UnlockTableOperation.java 8b70e06ca6 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/creation/package-info.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/info/package-info.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/lock/ShowLocksDesc.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ddl/table/lock/ShowLocksOperation.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 7c9d910c20 
  ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadTable.java b335f199f8 
  ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 24fc0d5437 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbLockManager.java 1a042783b0 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java 800d80a067 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManager.java bb46bf9490 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManagerImpl.java 0e148ed396 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java c9dd8541c2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/AcidExportSemanticAnalyzer.java 960dd34a8c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java bd6ae715d6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 07b40c996f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java c4e6e5cc53 
  ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java 4c1e2a27cb 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0e58fe20b4 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 67d27cdd87 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/DropTableHandler.java edef74e618 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TruncatePartitionHandler.java 05a9f9123f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TruncateTableHandler.java 5ef66fafa4 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AbortTxnsDesc.java e15d21549f 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateOrDropTriggerToPoolMappingDesc.java ce7589492f 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 1901defa27 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java dd3af1b35c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/KillQueryDesc.java 3a13064411 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 5d4e93e74a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 15e922ebd5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ShowCompactionsDesc.java 093800f6bb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ShowLocksDesc.java 2974cd9ac0 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ShowTxnsDesc.java 8696aaa74c 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java aa12ddb6af 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestHiveDecimalParse.java ae22b7f47a 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBCompact.java 90df85c58e 
  ql/src/test/queries/clientpositive/dbtxnmgr_abort.q d9239958f0 
  ql/src/test/queries/clientpositive/dbtxnmgr_showlocks.q 24a42ea7bc 
  ql/src/test/results/clientnegative/lockneg4.q.out c31a23c152 
  ql/src/test/results/clientnegative/lockneg5.q.out 674fad0d77 
  ql/src/test/results/clientpositive/dbtxnmgr_abort.q.out b1c808a302 
  ql/src/test/results/clientpositive/dbtxnmgr_showlocks.q.out aaf1c7835f 
  ql/src/test/results/clientpositive/kill_query.q.out 32c3f6ac57 
  ql/src/test/results/clientpositive/llap/kill_query.q.out 32c3f6ac57 


Diff: https://reviews.apache.org/r/70597/diff/1/


Testing
-------

All the unit tests are running fine + modified some to cover missing cases.


Thanks,

Miklos Gergely