You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Sebastian Schelter (JIRA)" <ji...@apache.org> on 2010/08/13 15:18:19 UTC
[jira] Created: (MAHOUT-477) SimilarityMatrixEntryKeyPartitioner
sometimes produces illegal partition numbers
SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers
--------------------------------------------------------------------------------
Key: MAHOUT-477
URL: https://issues.apache.org/jira/browse/MAHOUT-477
Project: Mahout
Issue Type: Bug
Components: Collaborative Filtering
Reporter: Sebastian Schelter
Priority: Minor
My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.
java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-477) SimilarityMatrixEntryKeyPartitioner
sometimes produces illegal partition numbers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898456#action_12898456 ]
Hudson commented on MAHOUT-477:
-------------------------------
Integrated in Mahout-Quality #186 (See [http://hudson.zones.apache.org/hudson/job/Mahout-Quality/186/])
MAHOUT-477
> SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers
> --------------------------------------------------------------------------------
>
> Key: MAHOUT-477
> URL: https://issues.apache.org/jira/browse/MAHOUT-477
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Assignee: Sean Owen
> Priority: Minor
> Fix For: 0.4
>
> Attachments: MAHOUT-477.patch
>
>
> My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.
> java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-477) SimilarityMatrixEntryKeyPartitioner
sometimes produces illegal partition numbers
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter updated MAHOUT-477:
--------------------------------------
Status: Patch Available (was: Open)
I had used a stupid custom Partitioner, simply switching to HashPartitioner did the trick.
> SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers
> --------------------------------------------------------------------------------
>
> Key: MAHOUT-477
> URL: https://issues.apache.org/jira/browse/MAHOUT-477
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Priority: Minor
> Attachments: MAHOUT-477.patch
>
>
> My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.
> java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-477) SimilarityMatrixEntryKeyPartitioner
sometimes produces illegal partition numbers
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated MAHOUT-477:
-----------------------------
Status: Resolved (was: Patch Available)
Assignee: Sean Owen
Fix Version/s: 0.4
Resolution: Fixed
OK I even removed that partitioner class since it's now just a HashPartitioner
> SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers
> --------------------------------------------------------------------------------
>
> Key: MAHOUT-477
> URL: https://issues.apache.org/jira/browse/MAHOUT-477
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Assignee: Sean Owen
> Priority: Minor
> Fix For: 0.4
>
> Attachments: MAHOUT-477.patch
>
>
> My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.
> java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-477) SimilarityMatrixEntryKeyPartitioner
sometimes produces illegal partition numbers
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter updated MAHOUT-477:
--------------------------------------
Attachment: MAHOUT-477.patch
> SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers
> --------------------------------------------------------------------------------
>
> Key: MAHOUT-477
> URL: https://issues.apache.org/jira/browse/MAHOUT-477
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Priority: Minor
> Attachments: MAHOUT-477.patch
>
>
> My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.
> java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.