You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Vaibhav Gumashta <vg...@hortonworks.com> on 2015/08/11 08:44:38 UTC

Re: Review Request 35792: HIVE-10438 - Architecture for ResultSet Compression via external plugin

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35792/#review94821
-----------------------------------------------------------



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (line 2201)
<https://reviews.apache.org/r/35792/#comment149462>

    For HiveServer2 specific configs, we use the format hive.server2.*



service/src/java/org/apache/hive/service/cli/RowSetFactory.java (line 31)
<https://reviews.apache.org/r/35792/#comment149459>

    What happens to versions > V9?



service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java (line 76)
<https://reviews.apache.org/r/35792/#comment149562>

    We should log the cause on catching this exception.



service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 46)
<https://reviews.apache.org/r/35792/#comment149563>

    How would the client send this string? Will it be a new API call from the driver code? Can you elaborate a bit more?



service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 59)
<https://reviews.apache.org/r/35792/#comment149465>

    Nit: whitespace.



service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 96)
<https://reviews.apache.org/r/35792/#comment149564>

    How is CompressorInfo being set?



service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 97)
<https://reviews.apache.org/r/35792/#comment149466>

    Nit: whitespace.



service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java (line 64)
<https://reviews.apache.org/r/35792/#comment149533>

    Nit: whitespace.



service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java (line 56)
<https://reviews.apache.org/r/35792/#comment149534>

    Nit: whitespace.


- Vaibhav Gumashta


On June 29, 2015, 5:41 p.m., Rohit Dholakia wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35792/
> -----------------------------------------------------------
> 
> (Updated June 29, 2015, 5:41 p.m.)
> 
> 
> Review request for hive, Vaibhav Gumashta, Xiaojian Wang, Xiao Meng, and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch enables ResultSet compression for Hive using external plugins. The patch proposes a plugin architecture that enables using external plugins to compress ResultSets on-the-fly.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 27f68df 
>   service/if/TCLIService.thrift baf583f 
>   service/src/gen/thrift/gen-cpp/TCLIService_types.h 7bceabd 
>   service/src/gen/thrift/gen-cpp/TCLIService_types.cpp 86eeea3 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TEnColumn.java PRE-CREATION 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementReq.java 4f157ad 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesReq.java c973fcc 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionReq.java c048161 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionResp.java 351f78b 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java a4279d2 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRowSet.java d16c8a4 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStatus.java 24a746e 
>   service/src/gen/thrift/gen-py/TCLIService/ttypes.py 068727c 
>   service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb b482533 
>   service/src/java/org/apache/hive/service/cli/Column.java 2e21f18 
>   service/src/java/org/apache/hive/service/cli/ColumnBasedSet.java 47a582e 
>   service/src/java/org/apache/hive/service/cli/RowSetFactory.java e8f68ea 
>   service/src/java/org/apache/hive/service/cli/compression/ColumnCompressor.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java dfb7faa 
>   service/src/main/resources/META-INF/services/org.apache.hive.service.cli.compression.ColumnCompressor PRE-CREATION 
>   service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java PRE-CREATION 
>   service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35792/diff/
> 
> 
> Testing
> -------
> 
> Testing has been done using a docker container-based query submitter that has an integer decompressor as part of it. Using the integer compressor (also provided) and the decompressor, the end-to-end functionality can be observed.
> 
> 
> File Attachments
> ----------------
> 
> Patch file
>   https://reviews.apache.org/media/uploaded/files/2015/06/23/16aa08f8-2393-460a-83ef-72464fc537db__HIVE-10438.patch
> 
> 
> Thanks,
> 
> Rohit Dholakia
> 
>


Re: Review Request 35792: HIVE-10438 - Architecture for ResultSet Compression via external plugin

Posted by Rohit Dholakia <ro...@simba.com>.

> On Aug. 11, 2015, 6:44 a.m., Vaibhav Gumashta wrote:
> > service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java, line 46
> > <https://reviews.apache.org/r/35792/diff/2/?file=994492#file994492line46>
> >
> >     How would the client send this string? Will it be a new API call from the driver code? Can you elaborate a bit more?

It won't be a new API call. It would be sent as part of TExecuteStatementReq. We updated the comments in EncodedColumnBasedSet to explain this.


- Rohit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35792/#review94821
-----------------------------------------------------------


On Aug. 15, 2015, 12:29 a.m., Rohit Dholakia wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35792/
> -----------------------------------------------------------
> 
> (Updated Aug. 15, 2015, 12:29 a.m.)
> 
> 
> Review request for hive, Vaibhav Gumashta, Xiaojian Wang, Xiao Meng, and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch enables ResultSet compression for Hive using external plugins. The patch proposes a plugin architecture that enables using external plugins to compress ResultSets on-the-fly.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 730f5be 
>   jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java bb2b695 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AbortTxnRequest.java 37e5bf1 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddDynamicPartitions.java 9c78c49 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java fcfaaf3 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java 9022019 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggrStats.java 917cec0 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AlreadyExistsException.java d7a317b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BinaryColumnStatsData.java 00b312d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BooleanColumnStatsData.java a0f3ab8 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CheckLockRequest.java 82e3031 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatistics.java 510dace 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatisticsDesc.java cf967b6 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatisticsObj.java 3c2b123 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CommitTxnRequest.java 5e8cd04 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CompactionRequest.java e9088e0 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ConfigValSecurityException.java 000670a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CurrentNotificationEventId.java d55d874 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 56b7281 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Date.java 0b406d6 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DateColumnStatsData.java 7a3d4ed 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Decimal.java 9215ce9 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java e64ca36 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java 2509ed5 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsExpr.java 5e3a2d1 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsRequest.java 24536ba 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsResult.java c139e65 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java 6accb8d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java ba69622 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventRequest.java f3d439c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventResponse.java d95ae06 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Function.java 50eff73 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetAllFunctionsResponse.java 0a9e27b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsInfoResponse.java 61dea16 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsResponse.java bd21de5 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetPrincipalsInRoleRequest.java 45e0cc1 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetPrincipalsInRoleResponse.java 805b898 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetRoleGrantsForPrincipalRequest.java c2c7259 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetRoleGrantsForPrincipalResponse.java 7156665 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokePrivilegeRequest.java 94e5a5e 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokePrivilegeResponse.java 1949aed 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokeRoleRequest.java ac3527a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokeRoleResponse.java 81ae31f 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatRequest.java 4a9ba7f 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeRequest.java dd6ed17 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeResponse.java 11bd82b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HiveObjectPrivilege.java 931b4e1 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HiveObjectRef.java 2573cea 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java 81227c3 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/IndexAlreadyExistsException.java 50517ed 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InsertEventRequestData.java 2e14cf9 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidInputException.java 87e2f6b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidObjectException.java 14dd7d8 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidOperationException.java 2e3f1e3 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidPartitionException.java 1d9b565 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockComponent.java 319f8bb 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockRequest.java 6894bfa 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockResponse.java 08acaeb 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java 93d2386 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/MetaException.java 883a1d4 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchLockException.java e214b36 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchObjectException.java 1292a64 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchTxnException.java d1c430d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEvent.java bcf4f51 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventRequest.java c2bc4e8 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventResponse.java 24f9ce4 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnRequest.java c5f9ccf 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnsResponse.java c233422 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Order.java da7bd55 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java 224d28e 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionListComposingSpec.java c50a100 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpec.java b5251af 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpecWithSharedSD.java 5574e0b 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionWithoutSD.java e1ec73e 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprRequest.java 6149c31 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprResult.java 740f7bd 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java 5d1ee87 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java da33014 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalPrivilegeSet.java 8f29f50 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrivilegeBag.java 2fd819c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrivilegeGrantInfo.java c04e196 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ResourceUri.java 3b3df25 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Role.java 5c882d2 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RolePrincipalGrant.java c4beb08 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java 8772180 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SerDeInfo.java b30e698 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java 7da298c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactRequest.java 7756384 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponse.java dd1e857 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponseElement.java cd7e79e 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksRequest.java 122c070 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponse.java 52b0bbc 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponseElement.java 8be9b05 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java bc64495 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java 165a879 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StringColumnStatsData.java 9906ff3 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java 51b9e38 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java 1edcaf9 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java 25a1f25 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java cdbae95 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnAbortedException.java a877338 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnInfo.java 8b255b9 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnOpenException.java 05af505 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Type.java 61e7ceb 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownDBException.java e05e79d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownPartitionException.java c626bf6 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownTableException.java 2856121 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnlockRequest.java cf248e0 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Version.java cc8d5f5 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote dc348ef 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Adjacency.java 5ea9b6e 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Graph.java e621cfa 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Operator.java 1b18aab 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java 5c5e0f8 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/QueryPlan.java d340d58 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Stage.java 7353933 
>   ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Task.java 09a2e7f 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/InnerStruct.java db325af 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1232ff9 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 4d2f5bf 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/IntString.java 23d7363 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 9447708 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MiniStruct.java d8c46f4 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 58498b0 
>   service/if/TCLIService.thrift baf583f 
>   service/src/gen/thrift/gen-cpp/TCLIService.h 29a9f4a 
>   service/src/gen/thrift/gen-cpp/TCLIService_types.h 4536b41 
>   service/src/gen/thrift/gen-cpp/TCLIService_types.cpp 742cfdc 
>   service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/HiveClusterStatus.java d4b6972 
>   service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/HiveServerException.java 760c81e 
>   service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/ThriftHive.java df793b1 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TArrayTypeEntry.java 5625516 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBinaryColumn.java 202399a 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBoolColumn.java 921e9de 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBoolValue.java 201c9fb 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TByteColumn.java cd9b6da 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TByteValue.java 42b5bd5 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCLIService.java 6bdd53d 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelDelegationTokenReq.java 1097869 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelDelegationTokenResp.java 5469108 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelOperationReq.java 83d191e 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelOperationResp.java b8d96df 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseOperationReq.java ca68866 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseOperationResp.java 092e7e4 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseSessionReq.java a5d910f 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseSessionResp.java ae4b554 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TColumnDesc.java e31aa81 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TDoubleColumn.java 7236d90 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TDoubleValue.java 4b0811e 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TEnColumn.java PRE-CREATION 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementReq.java feaed34 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementResp.java daf7b5c 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TFetchResultsReq.java 47d9a0b 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TFetchResultsResp.java b9f3ef6 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetCatalogsReq.java 0e63b9a 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetCatalogsResp.java 7067ff7 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetColumnsReq.java a153968 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetColumnsResp.java 49ecb98 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetDelegationTokenReq.java 2c2fa0f 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetDelegationTokenResp.java 4222005 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetFunctionsReq.java 5bc0540 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetFunctionsResp.java a64c948 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetInfoReq.java 7dd6a14 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetInfoResp.java 3e16318 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetOperationStatusReq.java f4d5fae 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetOperationStatusResp.java 897f7f5 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetResultSetMetadataReq.java 405ffc6 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetResultSetMetadataResp.java 3735310 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetSchemasReq.java 33088b4 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetSchemasResp.java ec598a3 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTableTypesReq.java 3e09a64 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTableTypesResp.java cdba9b5 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesReq.java 805e69f 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesResp.java 0d03dd8 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTypeInfoReq.java 0628e21 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTypeInfoResp.java 6058826 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/THandleIdentifier.java f20c4d0 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI16Column.java 267d109 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI16Value.java ddaf9ef 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI32Column.java c79df4f 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI32Value.java d521373 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI64Column.java 746ade2 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI64Value.java b419b80 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TMapTypeEntry.java 91a85ef 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionReq.java 657f868 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionResp.java 48f4b45 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOperationHandle.java db41117 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TPrimitiveTypeEntry.java f1c8d58 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java 6e714c6 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRenewDelegationTokenReq.java 91f8b00 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRenewDelegationTokenResp.java c01cc3f 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRow.java 197bab6 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRowSet.java cc1a148 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TSessionHandle.java 264e155 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStatus.java 1cd7980 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStringColumn.java d996529 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStringValue.java 9655f38 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStructTypeEntry.java d58184c 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTableSchema.java 796b73b 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTypeDesc.java 95b4466 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTypeQualifiers.java 15ac5a9 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TUnionTypeEntry.java eccf303 
>   service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TUserDefinedTypeEntry.java 21da61b 
>   service/src/gen/thrift/gen-py/TCLIService/ttypes.py efee8ef 
>   service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote 747c2be 
>   service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb bfb2b69 
>   service/src/java/org/apache/hive/service/cli/Column.java 2e21f18 
>   service/src/java/org/apache/hive/service/cli/ColumnBasedSet.java 47a582e 
>   service/src/java/org/apache/hive/service/cli/RowSetFactory.java e8f68ea 
>   service/src/java/org/apache/hive/service/cli/compression/ColumnCompressor.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 67bc778 
>   service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java PRE-CREATION 
>   service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java PRE-CREATION 
>   service/src/test/resources/META-INF/services/org.apache.hive.service.cli.compression.ColumnCompressor PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35792/diff/
> 
> 
> Testing
> -------
> 
> Testing has been done using a docker container-based query submitter that has an integer decompressor as part of it. Using the integer compressor (also provided) and the decompressor, the end-to-end functionality can be observed.
> 
> 
> File Attachments
> ----------------
> 
> Patch file
>   https://reviews.apache.org/media/uploaded/files/2015/06/23/16aa08f8-2393-460a-83ef-72464fc537db__HIVE-10438.patch
> 
> 
> Thanks,
> 
> Rohit Dholakia
> 
>