You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2018/10/17 18:57:04 UTC
[kudu-CR] WIP: Implement Spark DataSourceV2
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11708
Change subject: WIP: Implement Spark DataSourceV2
......................................................................
WIP: Implement Spark DataSourceV2
This is a rough Java implementation of Sparks
DataSourceV2 API. There is a lot of work to do yet,
but the base functionality exists. Given this is a
re-implementation, we should be extra carefull
to define the API we would like going forward.
It’s worth noting this patch would require dropping
Java 7 support.
Remaining things to do:
- Unify old and new API (remove duplicate code)
- Add credentials support
- Add timestamp propogation
- Add thorough logging
- Iron out public APIs/Interaction
- Add demo code/docs
- Add thorough testing
Change-Id: I0666c43cdddbb35ab235bd2b77f2e12351655923
---
M java/gradle.properties
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduPredicate.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/PartialRow.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RowResult.java
M java/kudu-spark/build.gradle
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/FilterMap.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/KuduConnector.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/RowMap.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/SchemaMap.java
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/DefaultSource.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduClientOptions.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataReader.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataReaderFactory.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataSourceReader.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataSourceWriter.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataWriter.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduDataWriterFactory.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduReadOptions.java
A java/kudu-spark/src/main/scala/org/apache/kudu/spark/sql/v2/KuduWriteOptions.java
A java/kudu-spark/src/test/scala/org/apache/kudu/spark/sql/v2/TestDefaultSource.java
23 files changed, 1,540 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/11708/1
--
To view, visit http://gerrit.cloudera.org:8080/11708
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0666c43cdddbb35ab235bd2b77f2e12351655923
Gerrit-Change-Number: 11708
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>