You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by cheng xu <ch...@intel.com> on 2015/02/12 02:46:52 UTC

Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/
-----------------------------------------------------------

Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.


Repository: hive-git


Description
-------

Changes includes:
* Update the DDLTask to support using statement
* Serialize the resource uri into table properties
* Deserialize the resource uri and add them to the session classloader
* Some query test and unit tests added


Diffs
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 062e520 
  itests/test-serde/pom.xml cb79072 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
  metastore/if/hive_metastore.thrift c2a2419 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 35a8a50 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp c7bf9ba 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 0f2fca8 
  metastore/src/gen/thrift/gen-php/metastore/Types.php e8afe4d 
  metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 81f70eb 
  metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 3208ecd 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
  ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
  ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
  ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/KryoHiveTest.java PRE-CREATION 
  ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
  ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java dda3c5f 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java ff0c1f2 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java fba49e4 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/PropValueUnion.java a50a508 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 334d225 

Diff: https://reviews.apache.org/r/30909/diff/


Testing
-------

newly added UT passed locally


Thanks,

cheng xu

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by cheng xu <ch...@intel.com>.


> On Feb. 12, 2015, 4:20 a.m., Jason Dere wrote:
> >

Thank you for your comments. I have an immature patch to share. Please see whether it is in the right direction. I will work on it after one 7 days' vacation.


- cheng


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/#review72106
-----------------------------------------------------------


On Feb. 17, 2015, 2:53 a.m., cheng xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30909/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 2:53 a.m.)
> 
> 
> Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Changes includes:
> 1.Update the DDLTask to support using statement
> 2.Serialize the resource uri into table properties
> 3.Deserialize the resource uri and add them to the session classloader
> 4.Some query test and unit tests added
> 
> 
> Diffs
> -----
> 
>   data/scripts/q_test_init.sql 0c8668c 
>   itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67 
>   itests/qtest/hive-it-test-serde-1.2.0-SNAPSHOT.jar PRE-CREATION 
>   itests/test-serde/pom.xml cb79072 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
>   metastore/if/hive_metastore.thrift c2a2419 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h dfa4f2a 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp f0b0559 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp b33e40b 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 35a8a50 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp c7bf9ba 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h f258266 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 7d8e0e2 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java ced8d4a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java 86c6fa2 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggrStats.java 8e0f757 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatistics.java a695645 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsResult.java 156338a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java ab0b399 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventRequest.java e2dc5af 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Function.java 182fbf4 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsInfoResponse.java a84c0cb 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsResponse.java 23dea5b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeResponse.java 6f5327c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java cb3a435 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InsertEventRequestData.java 5a9de92 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockRequest.java 7f70f98 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventResponse.java f61a49e 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnsResponse.java f369ab2 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java 242d54d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionListComposingSpec.java a634f55 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpecWithSharedSD.java 73d3067 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionWithoutSD.java 112b723 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprResult.java b459058 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java f515bc4 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java bf0eb38 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RequestPartsSpec.java 1da4f0c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java d130ba8 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java a7cf0a7 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponse.java d642edd 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponse.java 9a86a88 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java b91cc1c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageHandler.java PRE-CREATION 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java 229a819 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java bd997b4 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java 68ecd7d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java c3e9829 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 0f2fca8 
>   metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 1899457 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php e8afe4d 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote c257622 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 901bd1c 
>   metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 81f70eb 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 531c590 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 3208ecd 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 48bece8 
>   metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 6857a72 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 26ca208 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0ff2863 
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0aa0f51 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 2b49eab 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageHandler.java PRE-CREATION 
>   metastore/src/model/package.jdo b41b3d8 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java cf068e4 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5f28d73 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 3a2a6ee 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 9d5730d 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4aac39a 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
>   ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/KryoHiveTest.java PRE-CREATION 
>   ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
>   ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java dda3c5f 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java ff0c1f2 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java fba49e4 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/PropValueUnion.java a50a508 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 334d225 
> 
> Diff: https://reviews.apache.org/r/30909/diff/
> 
> 
> Testing
> -------
> 
> newly added UT passed locally
> 
> 
> Thanks,
> 
> cheng xu
> 
>

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by Jason Dere <jd...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/#review72106
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
<https://reviews.apache.org/r/30909/#comment118050>

    Why serialize here, would a string of comma-separated values work?



ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
<https://reviews.apache.org/r/30909/#comment118049>

    Just make sure none of these methods need to get called during the map/reduce tasks, since hive-metastore.jar (and thus the ResourceUri class) is not shipped during the map/reduce tasks. Might want to check this on a cluster just to make sure.


- Jason Dere


On Feb. 12, 2015, 1:49 a.m., cheng xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30909/
> -----------------------------------------------------------
> 
> (Updated Feb. 12, 2015, 1:49 a.m.)
> 
> 
> Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Changes includes:
> 1.Update the DDLTask to support using statement
> 2.Serialize the resource uri into table properties
> 3.Deserialize the resource uri and add them to the session classloader
> 4.Some query test and unit tests added
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 062e520 
>   itests/test-serde/pom.xml cb79072 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
>   metastore/if/hive_metastore.thrift c2a2419 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 35a8a50 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp c7bf9ba 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 0f2fca8 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php e8afe4d 
>   metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 81f70eb 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 3208ecd 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
>   ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/KryoHiveTest.java PRE-CREATION 
>   ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
>   ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java dda3c5f 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java ff0c1f2 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java fba49e4 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/PropValueUnion.java a50a508 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 334d225 
> 
> Diff: https://reviews.apache.org/r/30909/diff/
> 
> 
> Testing
> -------
> 
> newly added UT passed locally
> 
> 
> Thanks,
> 
> cheng xu
> 
>

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by cheng xu <ch...@intel.com>.


> On Feb. 17, 2015, 8:20 p.m., Brock Noland wrote:
> > Hey Ferdinand,
> > 
> > Generally I think the approach is sound. That is adding a permanent storage handler/serdes will make this more useful to users. It's unclear of the current code only handlers storage handlers or SERDEs as well?
> > 
> > One item we should change is the API signature of the new thrift HMS calls. All new HMS API's should be in the request/response format. For example if we have an API called "Some" the method should be:
> > 
> > SomeResponse Some(SomeRequest)
> > 
> > We should also stay away from ENUM's as they are not compatbile across releases.
> > 
> > Thanks you!!
> > 
> > Brock

Thanks Brock for your comments. I will update my patch after vacations.


- cheng


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/#review72782
-----------------------------------------------------------


On Feb. 17, 2015, 3:22 a.m., cheng xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30909/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 3:22 a.m.)
> 
> 
> Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Changes includes:
> 1.Update the DDLTask to support using statement
> 2.Serialize the resource uri into table properties
> 3.Deserialize the resource uri and add them to the session classloader
> 4.Some query test and unit tests added
> 
> 
> Diffs
> -----
> 
>   itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67 
>   itests/test-serde/pom.xml cb79072 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
>   metastore/if/hive_metastore.thrift c2a2419 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 26ca208 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0ff2863 
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0aa0f51 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 2b49eab 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageHandler.java PRE-CREATION 
>   metastore/src/model/package.jdo b41b3d8 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java cf068e4 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5f28d73 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 3a2a6ee 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 9d5730d 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4aac39a 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
>   ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
>   ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
>   ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30909/diff/
> 
> 
> Testing
> -------
> 
> newly added UT passed locally
> 
> 
> Thanks,
> 
> cheng xu
> 
>

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by Brock Noland <br...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/#review72782
-----------------------------------------------------------


Hey Ferdinand,

Generally I think the approach is sound. That is adding a permanent storage handler/serdes will make this more useful to users. It's unclear of the current code only handlers storage handlers or SERDEs as well?

One item we should change is the API signature of the new thrift HMS calls. All new HMS API's should be in the request/response format. For example if we have an API called "Some" the method should be:

SomeResponse Some(SomeRequest)

We should also stay away from ENUM's as they are not compatbile across releases.

Thanks you!!

Brock

- Brock Noland


On Feb. 17, 2015, 3:22 a.m., cheng xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30909/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 3:22 a.m.)
> 
> 
> Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Changes includes:
> 1.Update the DDLTask to support using statement
> 2.Serialize the resource uri into table properties
> 3.Deserialize the resource uri and add them to the session classloader
> 4.Some query test and unit tests added
> 
> 
> Diffs
> -----
> 
>   itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67 
>   itests/test-serde/pom.xml cb79072 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
>   itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
>   metastore/if/hive_metastore.thrift c2a2419 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 26ca208 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0ff2863 
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0aa0f51 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 2b49eab 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageHandler.java PRE-CREATION 
>   metastore/src/model/package.jdo b41b3d8 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java cf068e4 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5f28d73 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 3a2a6ee 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 9d5730d 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4aac39a 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
>   ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
>   ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
>   ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30909/diff/
> 
> 
> Testing
> -------
> 
> newly added UT passed locally
> 
> 
> Thanks,
> 
> cheng xu
> 
>

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by cheng xu <ch...@intel.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/
-----------------------------------------------------------

(Updated Feb. 17, 2015, 3:22 a.m.)


Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.


Changes
-------

Remove generated codes.


Repository: hive-git


Description
-------

Changes includes:
1.Update the DDLTask to support using statement
2.Serialize the resource uri into table properties
3.Deserialize the resource uri and add them to the session classloader
4.Some query test and unit tests added


Diffs (updated)
-----

  itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67 
  itests/test-serde/pom.xml cb79072 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
  metastore/if/hive_metastore.thrift c2a2419 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 26ca208 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0ff2863 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0aa0f51 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 2b49eab 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageHandler.java PRE-CREATION 
  metastore/src/model/package.jdo b41b3d8 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java cf068e4 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5f28d73 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 3a2a6ee 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 9d5730d 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4aac39a 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
  ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
  ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
  ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
  ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/30909/diff/


Testing
-------

newly added UT passed locally


Thanks,

cheng xu

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by cheng xu <ch...@intel.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/
-----------------------------------------------------------

(Updated Feb. 17, 2015, 2:53 a.m.)


Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.


Changes
-------

Patch is not ready. Just for sharing purpose addressing Jason's comments.


Repository: hive-git


Description
-------

Changes includes:
1.Update the DDLTask to support using statement
2.Serialize the resource uri into table properties
3.Deserialize the resource uri and add them to the session classloader
4.Some query test and unit tests added


Diffs (updated)
-----

  data/scripts/q_test_init.sql 0c8668c 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67 
  itests/qtest/hive-it-test-serde-1.2.0-SNAPSHOT.jar PRE-CREATION 
  itests/test-serde/pom.xml cb79072 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
  metastore/if/hive_metastore.thrift c2a2419 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h dfa4f2a 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp f0b0559 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp b33e40b 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 35a8a50 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp c7bf9ba 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h f258266 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 7d8e0e2 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java ced8d4a 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java 86c6fa2 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggrStats.java 8e0f757 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatistics.java a695645 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsResult.java 156338a 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java ab0b399 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventRequest.java e2dc5af 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Function.java 182fbf4 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsInfoResponse.java a84c0cb 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsResponse.java 23dea5b 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeResponse.java 6f5327c 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java cb3a435 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InsertEventRequestData.java 5a9de92 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockRequest.java 7f70f98 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventResponse.java f61a49e 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnsResponse.java f369ab2 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java 242d54d 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionListComposingSpec.java a634f55 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpecWithSharedSD.java 73d3067 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionWithoutSD.java 112b723 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprResult.java b459058 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java f515bc4 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java bf0eb38 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RequestPartsSpec.java 1da4f0c 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java d130ba8 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java a7cf0a7 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponse.java d642edd 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponse.java 9a86a88 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java b91cc1c 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageHandler.java PRE-CREATION 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java 229a819 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java bd997b4 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java 68ecd7d 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java c3e9829 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 0f2fca8 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 1899457 
  metastore/src/gen/thrift/gen-php/metastore/Types.php e8afe4d 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote c257622 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 901bd1c 
  metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 81f70eb 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 531c590 
  metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 3208ecd 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 48bece8 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 6857a72 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 26ca208 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 0ff2863 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0aa0f51 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 2b49eab 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageHandler.java PRE-CREATION 
  metastore/src/model/package.jdo b41b3d8 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java cf068e4 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5f28d73 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 3a2a6ee 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 9d5730d 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4aac39a 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
  ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
  ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
  ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/KryoHiveTest.java PRE-CREATION 
  ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
  ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java dda3c5f 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java ff0c1f2 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java fba49e4 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/PropValueUnion.java a50a508 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 334d225 

Diff: https://reviews.apache.org/r/30909/diff/


Testing
-------

newly added UT passed locally


Thanks,

cheng xu

Re: Review Request 30909: HIVE-9252 Linking custom SerDe jar to table definition.

Posted by cheng xu <ch...@intel.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30909/
-----------------------------------------------------------

(Updated Feb. 12, 2015, 1:49 a.m.)


Review request for hive, Brock Noland, Dong Chen, Mohit Sabharwal, and Sergio Pena.


Repository: hive-git


Description (updated)
-------

Changes includes:
1.Update the DDLTask to support using statement
2.Serialize the resource uri into table properties
3.Deserialize the resource uri and add them to the session classloader
4.Some query test and unit tests added


Diffs
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 062e520 
  itests/test-serde/pom.xml cb79072 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestBase64TextOutputFormat.java PRE-CREATION 
  itests/test-serde/src/main/java/org/apache/hadoop/hive/storagehandler/TestStorageHandler.java PRE-CREATION 
  metastore/if/hive_metastore.thrift c2a2419 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 35a8a50 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp c7bf9ba 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 0f2fca8 
  metastore/src/gen/thrift/gen-php/metastore/Types.php e8afe4d 
  metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 81f70eb 
  metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 3208ecd 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java 9da3071 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 089bd94 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 7d72783 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java c4633f6 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 69a4545 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java 1ef6d1b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 149b788 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdb9204 
  ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 7723430 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 8cadb96 
  ql/src/java/org/apache/hadoop/hive/ql/util/SemanticAnalyzerHelper.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 69f8889 
  ql/src/test/org/apache/hadoop/hive/ql/exec/persistence/KryoHiveTest.java PRE-CREATION 
  ql/src/test/queries/clientpositive/storage_handler_link_external_jar.q PRE-CREATION 
  ql/src/test/results/clientpositive/storage_handler_link_external_jar.q.out PRE-CREATION 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java dda3c5f 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java ff0c1f2 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java fba49e4 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/PropValueUnion.java a50a508 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 334d225 

Diff: https://reviews.apache.org/r/30909/diff/


Testing
-------

newly added UT passed locally


Thanks,

cheng xu