You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Pat Ferrel (JIRA)" <ji...@apache.org> on 2014/07/11 00:04:05 UTC
[jira] [Reopened] (MAHOUT-1464) Cooccurrence Analysis on Spark
[ https://issues.apache.org/jira/browse/MAHOUT-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pat Ferrel reopened MAHOUT-1464:
--------------------------------
Argh, I finally got to looking at cross-cooccurrence with matrices of different dimensions. This made a rather easy to fix but nasty error obvious.
If A is the primary self-similairty matrix we want to do A'A with, and B is the secondary matrix, we want to do A'B with it since for use in a recommender we want rows of the cooccurrence to be IDed by the columns of A (items in A), right?
The code does:
{code}
// Compute cross-co-occurrence matrix B'A
val drmBtA = drmB.t %*% drmA
{code}
it should do:
{code}
// Compute cross-co-occurrence matrix A'B
val drmAtB = drmA.t %*% drmB
{code}
Ted, can you confirm this? It's the way I've got it burned into my brain and applying ID translation with completely different IDs in B seems to confirm it.
> Cooccurrence Analysis on Spark
> ------------------------------
>
> Key: MAHOUT-1464
> URL: https://issues.apache.org/jira/browse/MAHOUT-1464
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Environment: hadoop, spark
> Reporter: Pat Ferrel
> Assignee: Pat Ferrel
> Fix For: 1.0
>
> Attachments: MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch, MAHOUT-1464.patch, run-spark-xrsj.sh
>
>
> Create a version of Cooccurrence Analysis (RowSimilarityJob with LLR) that runs on Spark. This should be compatible with Mahout Spark DRM DSL so a DRM can be used as input.
> Ideally this would extend to cover MAHOUT-1422. This cross-cooccurrence has several applications including cross-action recommendations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)