A problem about RowSimilarityJob

[刘鎏 <li...@gmail.com>]

Hi, all

I notice that in RowWeightMapper of RowSimilarityJob, if the input vector is
sparsevector, only non-zero column is write out. If the input matrix is
like:

          user1, user2
item1   1        0
item2   0        1

then output of RowWeightMapper should be {user1:item1} and {user2:item2}.

So how could I calculate the similarity of item1 and item2??

Thank you for your replies!

==================
        @Override
        protected void map(IntWritable row, VectorWritable vectorWritable,
                Context ctx) throws IOException, InterruptedException {

            Vector v = vectorWritable.get();
            double weight = similarity.weight(v);

            Iterator<Vector.Element> elementsIterator = v.iterateNonZero();
            while (elementsIterator.hasNext()) {
                Vector.Element element = elementsIterator.next();
                int column = element.index();
                double value = element.get();
                ctx.write(new VarIntWritable(column), new
WeightedOccurrence(
                        row.get(), value, weight));
            }
        }

===============



--

Re: A problem about RowSimilarityJob

[Sebastian Schelter <ss...@apache.org>]

Hi,

those items are by definition not similar as there is no user who 
interacted with both of them.

RowSimilarityJob is designed not to look at such pairs in order to 
minimize the number of item pairs that must be compared.

--sebastian


On 10.03.2011 14:14, 刘鎏 wrote:
> Hi, all
>
> I notice that in RowWeightMapper of RowSimilarityJob, if the input vector is
> sparsevector, only non-zero column is write out. If the input matrix is
> like:
>
>            user1, user2
> item1   1        0
> item2   0        1
>
> then output of RowWeightMapper should be {user1:item1} and {user2:item2}.
>
> So how could I calculate the similarity of item1 and item2??
>
> Thank you for your replies!
>
> ==================
>          @Override
>          protected void map(IntWritable row, VectorWritable vectorWritable,
>                  Context ctx) throws IOException, InterruptedException {
>
>              Vector v = vectorWritable.get();
>              double weight = similarity.weight(v);
>
>              Iterator<Vector.Element>  elementsIterator = v.iterateNonZero();
>              while (elementsIterator.hasNext()) {
>                  Vector.Element element = elementsIterator.next();
>                  int column = element.index();
>                  double value = element.get();
>                  ctx.write(new VarIntWritable(column), new
> WeightedOccurrence(
>                          row.get(), value, weight));
>              }
>          }
>
> ===============
>
>
>
> --
>