You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Hui Wen Han (JIRA)" <ji...@apache.org> on 2010/03/31 12:18:27 UTC
[jira] Created: (MAHOUT-354) make the output of RecommenderJob more
readable
make the output of RecommenderJob more readable
-----------------------------------------------
Key: MAHOUT-354
URL: https://issues.apache.org/jira/browse/MAHOUT-354
Project: Mahout
Issue Type: Improvement
Components: Collaborative Filtering
Affects Versions: 0.4
Reporter: Hui Wen Han
now the output of RecommenderMapper as following:
output.collect(userID, new RecommendedItemsWritable(recommendations));
Can we change it more readable like following:
private final Text user = new Text();
private final Text recomScore = new Text();
private static final String FIELD_SEPERATOR = ",";
for (RecommendedItem recommendation : recommendations)
{
user.set(String.valueOf(userId));
recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
output.collect(user, recomScore);
}
then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-354.
------------------------------
Assignee: Sean Owen
Fix Version/s: 0.4
Resolution: Fixed
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Assignee: Sean Owen
> Fix For: 0.4
>
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-354) make the output of RecommenderJob more
readable
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-354:
-------------------------------
Attachment: screenshot-1.jpg
the output like this
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-354) make the output of
RecommenderJob more readable
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851900#action_12851900 ]
Hui Wen Han edited comment on MAHOUT-354 at 3/31/10 3:38 PM:
-------------------------------------------------------------
I will try it again
was (Author: huiwenhan):
I try it again
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852318#action_12852318 ]
Sean Owen commented on MAHOUT-354:
----------------------------------
Try commenting out
jobConf.setBoolean("mapred.output.compress", true);
In AbstractJob. I am making this settable from the command line.
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851833#action_12851833 ]
Hui Wen Han commented on MAHOUT-354:
------------------------------------
I already comment line following file for test.
recommenderConf.setClass("mapred.output.compression.codec", GzipCodec.class, CompressionCodec.class);
JobConf recommenderConf = AbstractJob.prepareJobConf(userVectorPath, outputPath, jarFile,
SequenceFileInputFormat.class, RecommenderMapper.class, LongWritable.class,
RecommendedItemsWritable.class, IdentityReducer.class, LongWritable.class,
RecommendedItemsWritable.class, TextOutputFormat.class);
I am not sure TextOutputFormat compatible with RecommendedItemsWritable.class.
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851900#action_12851900 ]
Hui Wen Han commented on MAHOUT-354:
------------------------------------
I try it again
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851824#action_12851824 ]
Sean Owen commented on MAHOUT-354:
----------------------------------
You're dumping the compressed file as raw bytes -- uncompress it to see the textual representation.
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851866#action_12851866 ]
Sean Owen commented on MAHOUT-354:
----------------------------------
Hm, there's no reason it should not work with a text output format. It implements toString(), and It works for me as intended. Are you sure you're looking at the correct file, and the ones generated by a run where compression is disabled?
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851959#action_12851959 ]
Hui Wen Han commented on MAHOUT-354:
------------------------------------
I run the test several times,It 's also can not readable,seems binary.
I already commented compression line.
if (usersFile != null) {
recommenderConf.set(RecommenderMapper.USERS_FILE, usersFile);
}
//recommenderConf.setClass("mapred.output.compression.codec", GzipCodec.class, CompressionCodec.class);
JobClient.runJob(recommenderConf);
It's very weird.
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Attachments: screenshot-1.jpg
>
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-354) make the output of RecommenderJob
more readable
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851813#action_12851813 ]
Sean Owen commented on MAHOUT-354:
----------------------------------
I understand your point, though I'm reluctant to express the output this way. There is one, ordered list of recommendations per user. Expressing it this way says they are effectively a "set" of unordered and unrelated recommendations, which isn't quite right.
It's also not guaranteed that recommendations for one user will appear together this way, which could be confusing.
Does it make anything meaningfully easier to parse? it's already human-readable, and not a binary format.
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
> Key: MAHOUT-354
> URL: https://issues.apache.org/jira/browse/MAHOUT-354
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
>
> now the output of RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
> private final Text user = new Text();
> private final Text recomScore = new Text();
> private static final String FIELD_SEPERATOR = ",";
> for (RecommendedItem recommendation : recommendations)
> {
> user.set(String.valueOf(userId));
> recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
> output.collect(user, recomScore);
> }
> then user can read and verify the result more convenient and need not depend on the mahout API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.