You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by jerryshao <gi...@git.apache.org> on 2016/08/25 08:13:45 UTC
[GitHub] spark pull request #14804: [MINOR][Web UI] Correctly convert bytes in web UI
GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/14804
[MINOR][Web UI] Correctly convert bytes in web UI
## What changes were proposed in this pull request?
should be 1024 based, not 1000.
## How was this patch tested?
Manually verified.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark correct-convert-bytes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14804.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14804
----
commit fe78ecc2156ff6e842bd22b6b4419f0219a860b6
Author: jerryshao <ss...@hortonworks.com>
Date: 2016-08-25T08:01:12Z
Correctly convert the bytes in UI
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14804
KB = 1000 bytes, KiB = 1024 bytes. According to the suffixes we're using, 1000 is correct at the moment. Is the display inconsistent with something else in the UI or logs?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14804
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14804
**[Test build #64410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64410/consoleFull)** for PR 14804 at commit [`fe78ecc`](https://github.com/apache/spark/commit/fe78ecc2156ff6e842bd22b6b4419f0219a860b6).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14804
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14804
**[Test build #72944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72944/testReport)** for PR 14804 at commit [`fe78ecc`](https://github.com/apache/spark/commit/fe78ecc2156ff6e842bd22b6b4419f0219a860b6).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14804
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14804
**[Test build #64410 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64410/consoleFull)** for PR 14804 at commit [`fe78ecc`](https://github.com/apache/spark/commit/fe78ecc2156ff6e842bd22b6b4419f0219a860b6).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14804
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72944/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/14804
Because in the log it shows Memory MB in 1024 based, while in the web UI it is 1000 based, so this is slightly different.
You could check `Utils#bytesToString`. I think we unify this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14804
Ugh, yeah that's wrong in the sense that we are not showing MB, but MiB. I'd favor fixing the labels here and in Utils.bytesToString?
Then again, I see that we will also parse input of "500kb" as if it's "500KiB", using 1024 not 1000. That's wrong too really. But fixing it means a bit of a behavior change. We can support "500KB" or "500kb" but it would now mean 500*1000 bytes not 500*1024.
Well, maybe best to be consistently wrong than inconsistently wrong. Anyone feel at all inclined to take the hit in behavior change or just leave it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14804
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64410/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14804
**[Test build #72944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72944/testReport)** for PR 14804 at commit [`fe78ecc`](https://github.com/apache/spark/commit/fe78ecc2156ff6e842bd22b6b4419f0219a860b6).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14804
Meh, it's still ambiguous and there's a defined way to disambiguate, so it's unfortunate, but I'm OK with a step towards consistency in any event.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/14804
I think [here](http://ux.stackexchange.com/questions/13815/files-size-units-kib-vs-kb-vs-kb) has a precise definition. AFAIK in Spark the conversion is 1024 based either KB, K, or kb, KiB is not so commonly used. And we usually treat everything as 1024 based, so it might not be so necessary to differentiate them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org