You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by guoxiaolongzte <gi...@git.apache.org> on 2017/09/06 07:34:23 UTC
[GitHub] spark pull request #19144: Modify the title, 'Records' instead of 'Input Siz...
GitHub user guoxiaolongzte opened a pull request:
https://github.com/apache/spark/pull/19144
Modify the title, 'Records' instead of 'Input Size'
## What changes were proposed in this pull request?
Spark Streaming is processing data should be record, so the title should be 'Records', and should not be 'Input Size'.
Fix before:
Fix after:
## How was this patch tested?
manual tests
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/guoxiaolongzte/spark streamingUI
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19144.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19144
----
commit 9f07a3142c33080a6f091cba5d7b91e1ea2a9f97
Author: guoxiaolong <gu...@zte.com.cn>
Date: 2017-09-06T07:21:28Z
Modify the title, 'Recodes' instead of 'Input Size'
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
That's right, please look at https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/ui/AllBatchesTable.scala#L28
BTW, sometimes there's another reason you might not see the latest even on github.com/apache. Sometimes the ASF-Github sync stops. You can always go look at the ASF git, which is the real repo of reference.
https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=10e37f6eb6819c9233830c0d97e8fd1c713be0f1
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
Thank you, I was wrong. I've seen it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: Modify the title, 'Records' instead of 'Input Size'
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
I get it, but I don't think we use the "Records" terminology consistently in Spark, so not sure this is a worthwhile change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19144: [UI][Streaming]Modify the title, 'Records' instea...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19144
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@HyukjinKwon @jerryshao
Help to review the code, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: Modify the title, 'Records' instead of 'Input Size'
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19144
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@zsxwing Help to review the code, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
If this isn't getting support, it's not worth pinging indefinitely, please.
That said, I'll merge this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19144
Because you didn't fetch the upstream I guess ..
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
Actually yeah I see lots of that in the user-facing docs. OK, it's reasonable. Are you sure it's a # of records?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
@zsxwing what do you think?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19144
**[Test build #3931 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3931/testReport)** for PR 19144 at commit [`9f07a31`](https://github.com/apache/spark/commit/9f07a3142c33080a6f091cba5d7b91e1ea2a9f97).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: Modify the title, 'Records' instead of 'Input Size'
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
StreamInputInfo data structure:
case class StreamInputInfo(
inputStreamId: Int, **numRecords: Long**, metadata: Map[String, Any] = Map.empty) {
require(numRecords >= 0, "numRecords must not be negative")
def metadataDescription: Option[String] =
metadata.get(StreamInputInfo.METADATA_KEY_DESCRIPTION).map(_.toString)
}
So we use the "Records" in Spark Streaming.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
I can be sure. I tested streaming on kafka and streaming on sql .Are both records.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19144
**[Test build #3931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3931/testReport)** for PR 19144 at commit [`9f07a31`](https://github.com/apache/spark/commit/9f07a3142c33080a6f091cba5d7b91e1ea2a9f97).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
why is it closed?I think this should be incorporated into the master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@zsxwing Help to review the code, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@srowen Can this PR pass through?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19144
This is merged to master ... https://github.com/apache/spark/commit/10e37f6eb6819c9233830c0d97e8fd1c713be0f1 ...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
https://github.com/guoxiaolongzte/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/ui/AllBatchesTable.scala
It's strange that I didn't see my revised record. why?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19144: [UI][Streaming]Modify the title, 'Records' instead of 'I...
Posted by guoxiaolongzte <gi...@git.apache.org>.
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@zsxwing Help to review the code, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org