You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Avery Ching (JIRA)" <ji...@apache.org> on 2012/09/06 22:37:07 UTC

[jira] [Created] (GIRAPH-321) Divide by 0 exception

Avery Ching created GIRAPH-321:
----------------------------------

             Summary: Divide by 0 exception
                 Key: GIRAPH-321
                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
             Project: Giraph
          Issue Type: Bug
            Reporter: Avery Ching


hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner \
  org.apache.giraph.examples.SimplePageRankVertex -w 1 \
  -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
-ip graph.txt
  -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
-op output

where graph.txt has the following content:

1 2:1.0 3:1.0 4:1.0
2 1:1.0 3:1.0
3 1:1.0
4 1:1.0 3:1.0

One of the mappers gives throws a divbyzero exception.

java.lang.ArithmeticException: / by zero
	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-321) Divide by 0 exception

Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450130#comment-13450130 ] 

Eli Reisman commented on GIRAPH-321:
------------------------------------

Ouch. Will check it out now.

                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-321) Divide by 0 exception

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451062#comment-13451062 ] 

Hudson commented on GIRAPH-321:
-------------------------------

Integrated in Giraph-trunk-Commit #190 (See [https://builds.apache.org/job/Giraph-trunk-Commit/190/])
    GIRAPH-321: Divide by 0 exception. (ereisman via aching) (Revision 1382184)

     Result = SUCCESS
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382184
Files : 
* /giraph/trunk/CHANGELOG
* /giraph/trunk/src/main/java/org/apache/giraph/graph/LocalityInfoSorter.java

                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-321-1.patch
>
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (GIRAPH-321) Divide by 0 exception

Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Avery Ching resolved GIRAPH-321.
--------------------------------

    Resolution: Fixed
      Assignee: Eli Reisman

Thanks Eli.  Committed.
                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-321-1.patch
>
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-321) Divide by 0 exception

Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451369#comment-13451369 ] 

Eli Reisman commented on GIRAPH-321:
------------------------------------

Moving right now, sorry I didn't catch this sooner. Will switch my mailing lists back to a personal email so I don't miss anything on the list as soon as I get there. Waiting in line at the ISP to give back my internet box right now...
                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-321-1.patch
>
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-321) Divide by 0 exception

Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Reisman updated GIRAPH-321:
-------------------------------

    Attachment: GIRAPH-321-1.patch

Should be all set here, if we have 0-size path list then any local paths gets appended to the front of the pathList for iteration purposes rather than some other hashed location, avoiding the modulo operation that was hitting the div by zero.

Sorry for the annoyance! Will check this out a bit more soon, please follow up if anyone sees any more trouble.
                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>         Attachments: GIRAPH-321-1.patch
>
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-321) Divide by 0 exception

Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/GIRAPH-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450807#comment-13450807 ] 

Eli Reisman commented on GIRAPH-321:
------------------------------------

Avery, this is also an important fix for situations where the path list is temporarily empty because every split found on the list turned out to be local and was therefore pulled out of the original list to be sorted to the front. When we try to find a hashed index to append the locality list back into the path list, the path list is actually empty and the div by zero occurs. I imagine there are a lot of single node cluster and pseudo situations where this will always be the case.

Thanks again, nice save!

                
> Divide by 0 exception
> ---------------------
>
>                 Key: GIRAPH-321
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-321
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>         Attachments: GIRAPH-321-1.patch
>
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner \
>   org.apache.giraph.examples.SimplePageRankVertex -w 1 \
>   -if org.apache.giraph.examples.LongDoubleFloatDoubleTextInputFormat
> -ip graph.txt
>   -of org.apache.giraph.examples.VertexWithDoubleValueFloatEdgeTextOutputFormat
> -op output
> where graph.txt has the following content:
> 1 2:1.0 3:1.0 4:1.0
> 2 1:1.0 3:1.0
> 3 1:1.0
> 4 1:1.0 3:1.0
> One of the mappers gives throws a divbyzero exception.
> java.lang.ArithmeticException: / by zero
> 	at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
> 	at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
> 	at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
> 	at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira