You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by ericl <gi...@git.apache.org> on 2016/06/29 23:07:02 UTC
[GitHub] spark pull request #13983: [SPARK-16021] Fill freed memory in test to help c...
GitHub user ericl opened a pull request:
https://github.com/apache/spark/pull/13983
[SPARK-16021] Fill freed memory in test to help catch correctness bugs
## What changes were proposed in this pull request?
This patches `MemoryAllocator` to fill clean and freed memory with known byte values. Memory filling is flag-enabled in test only by default.
## How was this patch tested?
Unit test that it's on in test.
cc @sameeragarwal
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ericl/spark spark-16021
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13983.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13983
----
commit ad707acfb7c822452563d671d29cd04a7357a025
Author: Eric Liang <ek...@databricks.com>
Date: 2016-06-29T23:04:47Z
Wed Jun 29 16:04:47 PDT 2016
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13983
**[Test build #61510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61510/consoleFull)** for PR 13983 at commit [`ad707ac`](https://github.com/apache/spark/commit/ad707acfb7c822452563d671d29cd04a7357a025).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13983
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13983: [SPARK-16021] Fill freed memory in test to help c...
Posted by tedyu <gi...@git.apache.org>.
Github user tedyu commented on a diff in the pull request:
https://github.com/apache/spark/pull/13983#discussion_r69842071
--- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java ---
@@ -58,4 +61,17 @@ public void overlappingCopyMemory() {
Assert.assertEquals((byte)i, data[i + 1]);
}
}
+
+ @Test
+ public void memoryDebugFillEnabledInTest() {
+ Assert.assertTrue(MemoryAllocator.MEMORY_DEBUG_FILL_ENABLED);
--- End diff --
This assertion fails in Jenkins.
Did you intend to set the flag to true in a static block at the beginning of the test ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13983: [SPARK-16021] Fill freed memory in test to help c...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13983#discussion_r69854030
--- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java ---
@@ -58,4 +61,17 @@ public void overlappingCopyMemory() {
Assert.assertEquals((byte)i, data[i + 1]);
}
}
+
+ @Test
+ public void memoryDebugFillEnabledInTest() {
+ Assert.assertTrue(MemoryAllocator.MEMORY_DEBUG_FILL_ENABLED);
--- End diff --
Yea I think the problem is that Maven doesn't have the property set.
cc @ericl we need to set that in Maven too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13983: [SPARK-16021] Fill freed memory in test to help c...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13983#discussion_r69854073
--- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java ---
@@ -58,4 +61,17 @@ public void overlappingCopyMemory() {
Assert.assertEquals((byte)i, data[i + 1]);
}
}
+
+ @Test
+ public void memoryDebugFillEnabledInTest() {
+ Assert.assertTrue(MemoryAllocator.MEMORY_DEBUG_FILL_ENABLED);
--- End diff --
Actually it's fixed by https://github.com/apache/spark/pull/14084
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13983: [SPARK-16021] Fill freed memory in test to help c...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/13983
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13983
Merging in master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13983
**[Test build #61510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61510/consoleFull)** for PR 13983 at commit [`ad707ac`](https://github.com/apache/spark/commit/ad707acfb7c822452563d671d29cd04a7357a025).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on the issue:
https://github.com/apache/spark/pull/13983
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13983: [SPARK-16021] Fill freed memory in test to help catch co...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13983
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61510/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org