You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by arunmahadevan <gi...@git.apache.org> on 2018/05/17 18:10:39 UTC
[GitHub] spark pull request #21355: SPARK-24308: Handle DataReaderFactory to InputPar...
GitHub user arunmahadevan opened a pull request:
https://github.com/apache/spark/pull/21355
SPARK-24308: Handle DataReaderFactory to InputPartition rename in left over classes
## What changes were proposed in this pull request?
SPARK-24073 renames DataReaderFactory -> InputPartition and DataReader -> InputPartitionReader. Some classes still reflects the old name and causes confusion. This patch renames the left over classes to reflect the new interface and fixes a few comments.
## How was this patch tested?
Existing unit tests.
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arunmahadevan/spark SPARK-24308
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21355.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21355
----
commit b362574730c542c06557de24ac6498bf9c46fe62
Author: Arun Mahadevan <ar...@...>
Date: 2018-05-17T18:02:13Z
SPARK-24308: Handle DataReaderFactory to InputPartition rename in left over classes
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21355
Thanks! Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21355
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90743/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21355
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by jose-torres <gi...@git.apache.org>.
Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21355
LGTM. Thanks for handling this!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: SPARK-24308: Handle DataReaderFactory to InputPartition ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21355
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21355: [SPARK-24308][SQL] Handle DataReaderFactory to In...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21355
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21355
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21355
**[Test build #90800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90800/testReport)** for PR 21355 at commit [`8bef67b`](https://github.com/apache/spark/commit/8bef67bcea11f4d5ba84bb64a9ee3222113d1a7e).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21355
**[Test build #90743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90743/testReport)** for PR 21355 at commit [`b362574`](https://github.com/apache/spark/commit/b362574730c542c06557de24ac6498bf9c46fe62).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21355
**[Test build #90800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90800/testReport)** for PR 21355 at commit [`8bef67b`](https://github.com/apache/spark/commit/8bef67bcea11f4d5ba84bb64a9ee3222113d1a7e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by arunmahadevan <gi...@git.apache.org>.
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21355
ping @rdblue , @gatorsmile , @cloud-fan
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21355
LGTM, cc @jose-torres
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21355
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90800/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21355: [SPARK-24308][SQL] Handle DataReaderFactory to InputPart...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21355
**[Test build #90743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90743/testReport)** for PR 21355 at commit [`b362574`](https://github.com/apache/spark/commit/b362574730c542c06557de24ac6498bf9c46fe62).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class KafkaContinuousInputPartition(`
* `case class RateStreamContinuousInputPartition(`
* `class MemoryStreamInputPartition(records: Array[UnsafeRow])`
* `class ContinuousMemoryStreamInputPartition(`
* `class RateStreamMicroBatchInputPartition(`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21355: [SPARK-24308][SQL] Handle DataReaderFactory to In...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21355#discussion_r189163516
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ContinuousMemoryStream.scala ---
@@ -41,11 +41,11 @@ import org.apache.spark.util.RpcUtils
/**
* The overall strategy here is:
- * * ContinuousMemoryStream maintains a list of records for each partition. addData() will
--- End diff --
why remove the star mark?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org