You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Vaibhav Gumashta <vg...@hortonworks.com> on 2015/08/11 08:44:38 UTC
Re: Review Request 35792: HIVE-10438 - Architecture for ResultSet
Compression via external plugin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35792/#review94821
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (line 2201)
<https://reviews.apache.org/r/35792/#comment149462>
For HiveServer2 specific configs, we use the format hive.server2.*
service/src/java/org/apache/hive/service/cli/RowSetFactory.java (line 31)
<https://reviews.apache.org/r/35792/#comment149459>
What happens to versions > V9?
service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java (line 76)
<https://reviews.apache.org/r/35792/#comment149562>
We should log the cause on catching this exception.
service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 46)
<https://reviews.apache.org/r/35792/#comment149563>
How would the client send this string? Will it be a new API call from the driver code? Can you elaborate a bit more?
service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 59)
<https://reviews.apache.org/r/35792/#comment149465>
Nit: whitespace.
service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 96)
<https://reviews.apache.org/r/35792/#comment149564>
How is CompressorInfo being set?
service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java (line 97)
<https://reviews.apache.org/r/35792/#comment149466>
Nit: whitespace.
service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java (line 64)
<https://reviews.apache.org/r/35792/#comment149533>
Nit: whitespace.
service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java (line 56)
<https://reviews.apache.org/r/35792/#comment149534>
Nit: whitespace.
- Vaibhav Gumashta
On June 29, 2015, 5:41 p.m., Rohit Dholakia wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35792/
> -----------------------------------------------------------
>
> (Updated June 29, 2015, 5:41 p.m.)
>
>
> Review request for hive, Vaibhav Gumashta, Xiaojian Wang, Xiao Meng, and Xuefu Zhang.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ResultSet compression for Hive using external plugins. The patch proposes a plugin architecture that enables using external plugins to compress ResultSets on-the-fly.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 27f68df
> service/if/TCLIService.thrift baf583f
> service/src/gen/thrift/gen-cpp/TCLIService_types.h 7bceabd
> service/src/gen/thrift/gen-cpp/TCLIService_types.cpp 86eeea3
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TEnColumn.java PRE-CREATION
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementReq.java 4f157ad
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesReq.java c973fcc
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionReq.java c048161
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionResp.java 351f78b
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java a4279d2
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRowSet.java d16c8a4
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStatus.java 24a746e
> service/src/gen/thrift/gen-py/TCLIService/ttypes.py 068727c
> service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb b482533
> service/src/java/org/apache/hive/service/cli/Column.java 2e21f18
> service/src/java/org/apache/hive/service/cli/ColumnBasedSet.java 47a582e
> service/src/java/org/apache/hive/service/cli/RowSetFactory.java e8f68ea
> service/src/java/org/apache/hive/service/cli/compression/ColumnCompressor.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java dfb7faa
> service/src/main/resources/META-INF/services/org.apache.hive.service.cli.compression.ColumnCompressor PRE-CREATION
> service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java PRE-CREATION
> service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/35792/diff/
>
>
> Testing
> -------
>
> Testing has been done using a docker container-based query submitter that has an integer decompressor as part of it. Using the integer compressor (also provided) and the decompressor, the end-to-end functionality can be observed.
>
>
> File Attachments
> ----------------
>
> Patch file
> https://reviews.apache.org/media/uploaded/files/2015/06/23/16aa08f8-2393-460a-83ef-72464fc537db__HIVE-10438.patch
>
>
> Thanks,
>
> Rohit Dholakia
>
>
Re: Review Request 35792: HIVE-10438 - Architecture for ResultSet
Compression via external plugin
Posted by Rohit Dholakia <ro...@simba.com>.
> On Aug. 11, 2015, 6:44 a.m., Vaibhav Gumashta wrote:
> > service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java, line 46
> > <https://reviews.apache.org/r/35792/diff/2/?file=994492#file994492line46>
> >
> > How would the client send this string? Will it be a new API call from the driver code? Can you elaborate a bit more?
It won't be a new API call. It would be sent as part of TExecuteStatementReq. We updated the comments in EncodedColumnBasedSet to explain this.
- Rohit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35792/#review94821
-----------------------------------------------------------
On Aug. 15, 2015, 12:29 a.m., Rohit Dholakia wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35792/
> -----------------------------------------------------------
>
> (Updated Aug. 15, 2015, 12:29 a.m.)
>
>
> Review request for hive, Vaibhav Gumashta, Xiaojian Wang, Xiao Meng, and Xuefu Zhang.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This patch enables ResultSet compression for Hive using external plugins. The patch proposes a plugin architecture that enables using external plugins to compress ResultSets on-the-fly.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 730f5be
> jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java bb2b695
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AbortTxnRequest.java 37e5bf1
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddDynamicPartitions.java 9c78c49
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java fcfaaf3
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java 9022019
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggrStats.java 917cec0
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AlreadyExistsException.java d7a317b
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BinaryColumnStatsData.java 00b312d
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BooleanColumnStatsData.java a0f3ab8
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CheckLockRequest.java 82e3031
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatistics.java 510dace
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatisticsDesc.java cf967b6
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatisticsObj.java 3c2b123
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CommitTxnRequest.java 5e8cd04
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CompactionRequest.java e9088e0
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ConfigValSecurityException.java 000670a
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CurrentNotificationEventId.java d55d874
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 56b7281
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Date.java 0b406d6
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DateColumnStatsData.java 7a3d4ed
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Decimal.java 9215ce9
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java e64ca36
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java 2509ed5
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsExpr.java 5e3a2d1
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsRequest.java 24536ba
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DropPartitionsResult.java c139e65
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java 6accb8d
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java ba69622
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventRequest.java f3d439c
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FireEventResponse.java d95ae06
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Function.java 50eff73
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetAllFunctionsResponse.java 0a9e27b
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsInfoResponse.java 61dea16
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetOpenTxnsResponse.java bd21de5
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetPrincipalsInRoleRequest.java 45e0cc1
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetPrincipalsInRoleResponse.java 805b898
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetRoleGrantsForPrincipalRequest.java c2c7259
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetRoleGrantsForPrincipalResponse.java 7156665
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokePrivilegeRequest.java 94e5a5e
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokePrivilegeResponse.java 1949aed
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokeRoleRequest.java ac3527a
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GrantRevokeRoleResponse.java 81ae31f
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatRequest.java 4a9ba7f
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeRequest.java dd6ed17
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatTxnRangeResponse.java 11bd82b
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HiveObjectPrivilege.java 931b4e1
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HiveObjectRef.java 2573cea
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java 81227c3
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/IndexAlreadyExistsException.java 50517ed
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InsertEventRequestData.java 2e14cf9
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidInputException.java 87e2f6b
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidObjectException.java 14dd7d8
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidOperationException.java 2e3f1e3
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/InvalidPartitionException.java 1d9b565
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockComponent.java 319f8bb
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockRequest.java 6894bfa
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LockResponse.java 08acaeb
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java 93d2386
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/MetaException.java 883a1d4
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchLockException.java e214b36
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchObjectException.java 1292a64
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NoSuchTxnException.java d1c430d
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEvent.java bcf4f51
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventRequest.java c2bc4e8
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/NotificationEventResponse.java 24f9ce4
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnRequest.java c5f9ccf
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OpenTxnsResponse.java c233422
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Order.java da7bd55
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java 224d28e
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionListComposingSpec.java c50a100
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpec.java b5251af
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionSpecWithSharedSD.java 5574e0b
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionWithoutSD.java e1ec73e
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprRequest.java 6149c31
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsByExprResult.java 740f7bd
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java 5d1ee87
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java da33014
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalPrivilegeSet.java 8f29f50
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrivilegeBag.java 2fd819c
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrivilegeGrantInfo.java c04e196
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ResourceUri.java 3b3df25
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Role.java 5c882d2
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RolePrincipalGrant.java c4beb08
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java 8772180
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SerDeInfo.java b30e698
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java 7da298c
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactRequest.java 7756384
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponse.java dd1e857
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowCompactResponseElement.java cd7e79e
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksRequest.java 122c070
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponse.java 52b0bbc
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ShowLocksResponseElement.java 8be9b05
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java bc64495
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java 165a879
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StringColumnStatsData.java 9906ff3
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java 51b9e38
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java 1edcaf9
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java 25a1f25
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java cdbae95
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnAbortedException.java a877338
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnInfo.java 8b255b9
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TxnOpenException.java 05af505
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Type.java 61e7ceb
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownDBException.java e05e79d
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownPartitionException.java c626bf6
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnknownTableException.java 2856121
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/UnlockRequest.java cf248e0
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Version.java cc8d5f5
> metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote dc348ef
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Adjacency.java 5ea9b6e
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Graph.java e621cfa
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Operator.java 1b18aab
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java 5c5e0f8
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/QueryPlan.java d340d58
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Stage.java 7353933
> ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Task.java 09a2e7f
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/InnerStruct.java db325af
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1232ff9
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 4d2f5bf
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/IntString.java 23d7363
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 9447708
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MiniStruct.java d8c46f4
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/SetIntString.java 58498b0
> service/if/TCLIService.thrift baf583f
> service/src/gen/thrift/gen-cpp/TCLIService.h 29a9f4a
> service/src/gen/thrift/gen-cpp/TCLIService_types.h 4536b41
> service/src/gen/thrift/gen-cpp/TCLIService_types.cpp 742cfdc
> service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/HiveClusterStatus.java d4b6972
> service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/HiveServerException.java 760c81e
> service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/ThriftHive.java df793b1
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TArrayTypeEntry.java 5625516
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBinaryColumn.java 202399a
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBoolColumn.java 921e9de
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TBoolValue.java 201c9fb
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TByteColumn.java cd9b6da
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TByteValue.java 42b5bd5
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCLIService.java 6bdd53d
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelDelegationTokenReq.java 1097869
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelDelegationTokenResp.java 5469108
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelOperationReq.java 83d191e
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCancelOperationResp.java b8d96df
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseOperationReq.java ca68866
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseOperationResp.java 092e7e4
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseSessionReq.java a5d910f
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCloseSessionResp.java ae4b554
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TColumnDesc.java e31aa81
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TDoubleColumn.java 7236d90
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TDoubleValue.java 4b0811e
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TEnColumn.java PRE-CREATION
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementReq.java feaed34
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementResp.java daf7b5c
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TFetchResultsReq.java 47d9a0b
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TFetchResultsResp.java b9f3ef6
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetCatalogsReq.java 0e63b9a
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetCatalogsResp.java 7067ff7
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetColumnsReq.java a153968
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetColumnsResp.java 49ecb98
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetDelegationTokenReq.java 2c2fa0f
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetDelegationTokenResp.java 4222005
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetFunctionsReq.java 5bc0540
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetFunctionsResp.java a64c948
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetInfoReq.java 7dd6a14
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetInfoResp.java 3e16318
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetOperationStatusReq.java f4d5fae
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetOperationStatusResp.java 897f7f5
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetResultSetMetadataReq.java 405ffc6
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetResultSetMetadataResp.java 3735310
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetSchemasReq.java 33088b4
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetSchemasResp.java ec598a3
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTableTypesReq.java 3e09a64
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTableTypesResp.java cdba9b5
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesReq.java 805e69f
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesResp.java 0d03dd8
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTypeInfoReq.java 0628e21
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTypeInfoResp.java 6058826
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/THandleIdentifier.java f20c4d0
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI16Column.java 267d109
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI16Value.java ddaf9ef
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI32Column.java c79df4f
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI32Value.java d521373
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI64Column.java 746ade2
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TI64Value.java b419b80
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TMapTypeEntry.java 91a85ef
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionReq.java 657f868
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOpenSessionResp.java 48f4b45
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TOperationHandle.java db41117
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TPrimitiveTypeEntry.java f1c8d58
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TProtocolVersion.java 6e714c6
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRenewDelegationTokenReq.java 91f8b00
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRenewDelegationTokenResp.java c01cc3f
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRow.java 197bab6
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TRowSet.java cc1a148
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TSessionHandle.java 264e155
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStatus.java 1cd7980
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStringColumn.java d996529
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStringValue.java 9655f38
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TStructTypeEntry.java d58184c
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTableSchema.java 796b73b
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTypeDesc.java 95b4466
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TTypeQualifiers.java 15ac5a9
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TUnionTypeEntry.java eccf303
> service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TUserDefinedTypeEntry.java 21da61b
> service/src/gen/thrift/gen-py/TCLIService/ttypes.py efee8ef
> service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote 747c2be
> service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb bfb2b69
> service/src/java/org/apache/hive/service/cli/Column.java 2e21f18
> service/src/java/org/apache/hive/service/cli/ColumnBasedSet.java 47a582e
> service/src/java/org/apache/hive/service/cli/RowSetFactory.java e8f68ea
> service/src/java/org/apache/hive/service/cli/compression/ColumnCompressor.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/compression/ColumnCompressorService.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/compression/EncodedColumnBasedSet.java PRE-CREATION
> service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 67bc778
> service/src/test/org/apache/hive/service/cli/compression/SnappyIntCompressor.java PRE-CREATION
> service/src/test/org/apache/hive/service/cli/compression/TestEncodedColumnBasedSet.java PRE-CREATION
> service/src/test/resources/META-INF/services/org.apache.hive.service.cli.compression.ColumnCompressor PRE-CREATION
>
> Diff: https://reviews.apache.org/r/35792/diff/
>
>
> Testing
> -------
>
> Testing has been done using a docker container-based query submitter that has an integer decompressor as part of it. Using the integer compressor (also provided) and the decompressor, the end-to-end functionality can be observed.
>
>
> File Attachments
> ----------------
>
> Patch file
> https://reviews.apache.org/media/uploaded/files/2015/06/23/16aa08f8-2393-460a-83ef-72464fc537db__HIVE-10438.patch
>
>
> Thanks,
>
> Rohit Dholakia
>
>