You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Hui Wen Han (JIRA)" <ji...@apache.org> on 2010/03/31 12:18:27 UTC

[jira] Created: (MAHOUT-354) make the output of RecommenderJob more readable

make the output of RecommenderJob more readable
-----------------------------------------------

                 Key: MAHOUT-354
                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
             Project: Mahout
          Issue Type: Improvement
          Components: Collaborative Filtering
    Affects Versions: 0.4
            Reporter: Hui Wen Han


now the output of  RecommenderMapper as following:
output.collect(userID, new RecommendedItemsWritable(recommendations));

Can we change it more readable like following:

   private final Text user = new Text();
    private final Text recomScore = new Text();
    private static final String FIELD_SEPERATOR = ",";
    for (RecommendedItem recommendation : recommendations)
    {
    	user.set(String.valueOf(userId));
    	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
        output.collect(user, recomScore);
    }

then user can read and verify the result more convenient and need not depend on the mahout  API


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved MAHOUT-354.
------------------------------

         Assignee: Sean Owen
    Fix Version/s: 0.4
       Resolution: Fixed

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>            Assignee: Sean Owen
>             Fix For: 0.4
>
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hui Wen Han updated MAHOUT-354:
-------------------------------

    Attachment: screenshot-1.jpg

the output like this

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851900#action_12851900 ] 

Hui Wen Han edited comment on MAHOUT-354 at 3/31/10 3:38 PM:
-------------------------------------------------------------

I will try it again

      was (Author: huiwenhan):
    I try it again
  
> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852318#action_12852318 ] 

Sean Owen commented on MAHOUT-354:
----------------------------------

Try commenting out       

jobConf.setBoolean("mapred.output.compress", true);

In AbstractJob. I am making this settable from the command line.

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851833#action_12851833 ] 

Hui Wen Han commented on MAHOUT-354:
------------------------------------

I already comment line following file for test.
recommenderConf.setClass("mapred.output.compression.codec", GzipCodec.class, CompressionCodec.class);

JobConf recommenderConf = AbstractJob.prepareJobConf(userVectorPath, outputPath, jarFile,
      SequenceFileInputFormat.class, RecommenderMapper.class, LongWritable.class,
      RecommendedItemsWritable.class, IdentityReducer.class, LongWritable.class,
      RecommendedItemsWritable.class, TextOutputFormat.class);

I am not sure TextOutputFormat compatible with  RecommendedItemsWritable.class.

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851900#action_12851900 ] 

Hui Wen Han commented on MAHOUT-354:
------------------------------------

I try it again

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851824#action_12851824 ] 

Sean Owen commented on MAHOUT-354:
----------------------------------

You're dumping the compressed file as raw bytes -- uncompress it to see the textual representation.

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851866#action_12851866 ] 

Sean Owen commented on MAHOUT-354:
----------------------------------

Hm, there's no reason it should not work with a text output format. It implements toString(), and It works for me as intended. Are you sure you're looking at the correct file, and the ones generated by a run where compression is disabled? 

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851959#action_12851959 ] 

Hui Wen Han commented on MAHOUT-354:
------------------------------------

I run the test several times,It 's also can not readable,seems  binary.

I already commented compression line.

    if (usersFile != null) {
      recommenderConf.set(RecommenderMapper.USERS_FILE, usersFile);
    }
    //recommenderConf.setClass("mapred.output.compression.codec", GzipCodec.class, CompressionCodec.class);
    JobClient.runJob(recommenderConf);

It's very weird.

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>         Attachments: screenshot-1.jpg
>
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAHOUT-354) make the output of RecommenderJob more readable

Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851813#action_12851813 ] 

Sean Owen commented on MAHOUT-354:
----------------------------------

I understand your point, though I'm reluctant to express the output this way. There is one, ordered list of recommendations per user. Expressing it this way says they are effectively a "set" of unordered and unrelated recommendations, which isn't quite right.

It's also not guaranteed that recommendations for one user will appear together this way, which could be confusing.

Does it make anything meaningfully easier to parse? it's already human-readable, and not a binary format.

> make the output of RecommenderJob more readable
> -----------------------------------------------
>
>                 Key: MAHOUT-354
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-354
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Collaborative Filtering
>    Affects Versions: 0.4
>            Reporter: Hui Wen Han
>
> now the output of  RecommenderMapper as following:
> output.collect(userID, new RecommendedItemsWritable(recommendations));
> Can we change it more readable like following:
>    private final Text user = new Text();
>     private final Text recomScore = new Text();
>     private static final String FIELD_SEPERATOR = ",";
>     for (RecommendedItem recommendation : recommendations)
>     {
>     	user.set(String.valueOf(userId));
>     	recomScore.set(recommendation.getItemID() + FIELD_SEPERATOR + recommendation.getValue());
>         output.collect(user, recomScore);
>     }
> then user can read and verify the result more convenient and need not depend on the mahout  API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.