You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Jakob Homan (Created) (JIRA)" <ji...@apache.org> on 2011/10/08 01:15:29 UTC
[jira] [Created] (GIRAPH-48) All tests failing under non-secure
build
All tests failing under non-secure build
----------------------------------------
Key: GIRAPH-48
URL: https://issues.apache.org/jira/browse/GIRAPH-48
Project: Giraph
Issue Type: Bug
Reporter: Jakob Homan
After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
{noformat}mvn -Dhadoop=non_secure clean test{noformat}
results in:
{noformat}Failed tests:
testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
testBspSuperStep(org.apache.giraph.TestBspBasic)
testBspMsg(org.apache.giraph.TestBspBasic)
testBspCombiner(org.apache.giraph.TestBspBasic)
testBspPageRank(org.apache.giraph.TestBspBasic)
testBspShortestPaths(org.apache.giraph.TestBspBasic)
testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Jakob Homan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123926#comment-13123926 ]
Jakob Homan commented on GIRAPH-48:
-----------------------------------
+1 on the fix, although most of the patch appears to be whitespace changes...
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-48) All tests failing under non-secure
build
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123907#comment-13123907 ]
jiraposter@reviews.apache.org commented on GIRAPH-48:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2322/
-----------------------------------------------------------
Review request for giraph.
Summary
-------
9 unittests failed due to
java.lang.IllegalArgumentException
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:480)
at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:59)
at org.apache.giraph.comm.BasicRPCCommunications.<init>(BasicRPCCommunications.java:375)
at org.apache.giraph.comm.RPCCommunications.<init>(RPCCommunications.java:68)
at org.apache.giraph.graph.GraphMapper.map(GraphMapper.java:571)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
The issue is that with only a single worker, the ThreadpoolExecutor fails because of having 0 threads as an argument.
This addresses bug GIRAPH-48.
https://issues.apache.org/jira /browse/GIRAPH-48
Diffs
-----
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1180785
Diff: https://reviews.apache.org/r/2322/diff
Testing
-------
Passed unittests.
Thanks,
Avery
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123936#comment-13123936 ]
Hudson commented on GIRAPH-48:
------------------------------
Integrated in Giraph-trunk-Commit #15 (See [https://builds.apache.org/job/Giraph-trunk-Commit/15/])
GIRAPH-48: numFlushThreads is 0 when doing a single worker
unittest. Changing the minimum to 1. (aching)
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180805
Files :
* /incubator/giraph/trunk/CHANGELOG
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Avery Ching
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Avery Ching (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123935#comment-13123935 ]
Avery Ching commented on GIRAPH-48:
-----------------------------------
Yeah, Eclipse automatically does that. The main change is very small:
int numWorkers = conf.getInt(GiraphJob.MAX_WORKERS, numTasks);
- // if the number of flush threads is unset, it is set to
- // the number of max workers.
- int numFlushThreads = conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS, numWorkers-1);
+ // If the number of flush threads is unset, it is set to
+ // the number of max workers - 1 or a minimum of 1.
+ int numFlushThreads =
+ Math.max(conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS,
+ numWorkers - 1),
+ 1);
Thanks for the quick review! I've check it in.
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Avery Ching (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching resolved GIRAPH-48.
-------------------------------
Resolution: Fixed
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Avery Ching
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Avery Ching (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123912#comment-13123912 ]
Avery Ching commented on GIRAPH-48:
-----------------------------------
This issue affect all versions of Hadoop I believe. I tested with 0.20.203 and found the same issues. This fix passed both LocalJobRunner and local map-reduce instance tests with both 0.20.203 and non secure Hadoop.
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (GIRAPH-48) All tests failing under
non-secure build
Posted by "Avery Ching (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123935#comment-13123935 ]
Avery Ching edited comment on GIRAPH-48 at 10/10/11 7:40 AM:
-------------------------------------------------------------
Yeah, Eclipse automatically does that. The main change is very small:
int numWorkers = conf.getInt(GiraphJob.MAX_WORKERS, numTasks);
- // if the number of flush threads is unset, it is set to
- // the number of max workers.
- int numFlushThreads = conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS, numWorkers-1);
+ // If the number of flush threads is unset, it is set to
+ // the number of max workers - 1 or a minimum of 1.
+ int numFlushThreads =
+ Math.max(conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS,
+ numWorkers - 1),
+ 1);
Thanks for the quick review! I've checked it in.
was (Author: aching):
Yeah, Eclipse automatically does that. The main change is very small:
int numWorkers = conf.getInt(GiraphJob.MAX_WORKERS, numTasks);
- // if the number of flush threads is unset, it is set to
- // the number of max workers.
- int numFlushThreads = conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS, numWorkers-1);
+ // If the number of flush threads is unset, it is set to
+ // the number of max workers - 1 or a minimum of 1.
+ int numFlushThreads =
+ Math.max(conf.getInt(GiraphJob.MSG_NUM_FLUSH_THREADS,
+ numWorkers - 1),
+ 1);
Thanks for the quick review! I've check it in.
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Avery Ching (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching reassigned GIRAPH-48:
---------------------------------
Assignee: Avery Ching
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Avery Ching
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-48) All tests failing under non-secure
build
Posted by "Jakob Homan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated GIRAPH-48:
------------------------------
Priority: Critical (was: Major)
> All tests failing under non-secure build
> ----------------------------------------
>
> Key: GIRAPH-48
> URL: https://issues.apache.org/jira/browse/GIRAPH-48
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Priority: Critical
>
> After GIRAPH-12, quite a lot of tests are failing when run under non-secure hadoop:
> {noformat}mvn -Dhadoop=non_secure clean test{noformat}
> results in:
> {noformat}Failed tests:
> testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer)
> testBspCheckpoint(org.apache.giraph.TestManualCheckpoint)
> testBspSuperStep(org.apache.giraph.TestBspBasic)
> testBspMsg(org.apache.giraph.TestBspBasic)
> testBspCombiner(org.apache.giraph.TestBspBasic)
> testBspPageRank(org.apache.giraph.TestBspBasic)
> testBspShortestPaths(org.apache.giraph.TestBspBasic)
> testMutateGraph(org.apache.giraph.TestMutateGraphVertex)
> testContinue(org.apache.giraph.TestJsonBase64Format){noformat}
> rolling back to the previous commit, the tests pass.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira