You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by "Chao Sun (Code Review)" <ge...@cloudera.org> on 2016/05/06 21:18:21 UTC
[RecordServiceClient-CR](master_dev) Add complex type support for client side
Chao Sun has uploaded a new change for review.
http://gerrit.cloudera.org:8080/2988
Change subject: Add complex type support for client side
......................................................................
Add complex type support for client side
This adds complex type support for the client side (Java). All the
implementations here conform to the Apache Arrow's standard, with the
exception of:
- struct type is different in that it doesn't have a "is_null" array
associated with. This is due to how struct is handled in Impala, and
will be changed once that is fixed.
- no support for union type.
All the work are included under package com.cloudera.recordservice.core.data.
It is also accompanied with some some tests in TestArray class. Some of these
use the functional.complextypestbl on the Impala side.
The next step is to integrate this with input formats and frameworks.
Change-Id: Ia2c14c85eb9eeb06ae30d7feebf89170136235df
---
M api/RecordService.thrift
M cpp/samples/record-service-client.cc
M cpp/samples/rs-grep.cc
M cpp/tests/client-test.cc
M cpp/tests/test-common.h
M java/avro/src/main/java/com/cloudera/recordservice/avro/GenericRecords.java
M java/avro/src/main/java/com/cloudera/recordservice/avro/KeyValueRecords.java
M java/avro/src/main/java/com/cloudera/recordservice/avro/RecordUtil.java
M java/core/src/main/java/com/cloudera/recordservice/core/Records.java
M java/core/src/main/java/com/cloudera/recordservice/core/Schema.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/AbstractArray.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/Array.java
R java/core/src/main/java/com/cloudera/recordservice/core/data/ByteArray.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/CharArray.java
R java/core/src/main/java/com/cloudera/recordservice/core/data/Decimal.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/ListArray.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/ListArrayBase.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/Map.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/MapArray.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/PrimitiveArray.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/Record.java
A java/core/src/main/java/com/cloudera/recordservice/core/data/StructArray.java
R java/core/src/main/java/com/cloudera/recordservice/core/data/TimestampNanos.java
M java/core/src/test/java/com/cloudera/recordservice/core/SampleClient.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestBase.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestBasicClient.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestFaultTolerance.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestKerberosConnection.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestS3.java
M java/core/src/test/java/com/cloudera/recordservice/core/TestSchema.java
A java/core/src/test/java/com/cloudera/recordservice/core/data/TestComplexTypes.java
R java/core/src/test/java/com/cloudera/recordservice/core/data/TestUtil.java
M java/examples/src/main/java/com/cloudera/recordservice/examples/RSCat.java
M java/examples/src/main/java/com/cloudera/recordservice/examples/SumQueryBenchmark.java
M java/examples/src/main/java/com/cloudera/recordservice/examples/terasort/RecordServiceTeraInputFormat.java
M java/hcatalog-pig-adapter/src/main/java/com/cloudera/recordservice/pig/PigHCatUtil.java
M java/mr/src/main/java/com/cloudera/recordservice/mapred/TextInputFormat.java
M java/mr/src/main/java/com/cloudera/recordservice/mapreduce/RecordServiceInputFormat.java
M java/mr/src/main/java/com/cloudera/recordservice/mapreduce/TextInputFormat.java
M java/mr/src/main/java/com/cloudera/recordservice/mr/DecimalWritable.java
M java/mr/src/main/java/com/cloudera/recordservice/mr/RecordServiceRecord.java
M java/mr/src/main/java/com/cloudera/recordservice/mr/TimestampNanosWritable.java
M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRDD.scala
M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRecordRDD.scala
M java/spark/src/main/scala/com/cloudera/recordservice/spark/RecordServiceRelation.scala
M java/spark/src/main/scala/com/cloudera/recordservice/spark/SchemaRecordServiceRDD.scala
M java/spark/src/test/scala/com/cloudera/recordservice/spark/BasicClientSuite.scala
M tests/benchmarks.py
M tests/create-tbls.sql
M tests/load-test-data.sh
50 files changed, 2,274 insertions(+), 490 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/RecordServiceClient refs/changes/88/2988/1
--
To view, visit http://gerrit.cloudera.org:8080/2988
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia2c14c85eb9eeb06ae30d7feebf89170136235df
Gerrit-PatchSet: 1
Gerrit-Project: RecordServiceClient
Gerrit-Branch: master_dev
Gerrit-Owner: Chao Sun <su...@gmail.com>