You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Mark Kerzner <ma...@gmail.com> on 2011/01/28 00:52:34 UTC

Writing to HBase in Reduce

Hi,

this is how I write to a 0.20 HBase

        public void reduce(Text k2, Iterator<IntWritable> itrtr,
OutputCollector<ImmutableBytesWritable, BatchUpdate> oc, Reporter rprtr)
throws IOException {
            int sum = 0;
            while (itrtr.hasNext()) {
                IntWritable val = itrtr.next();
                sum += val.get();
            }
            String family = "stats:";
            String familyCell = family + k2.toString();
            oc.collect(new ImmutableBytesWritable(familyCell.getBytes()),
new BatchUpdate("" + sum));
        }

and the function is called, I step through it and see the correct values,
but then I see no rows in the output table. No error messages either.

Could you please help?

Thank you,
Mark

Re: Writing to HBase in Reduce

Posted by Mark Kerzner <ma...@gmail.com>.

V1 of the book does not write back to the HBase, and V2 does not have this
examples at all (I wonder why).

I have fixed the problem by using BatchUpdate in the right way:

            BatchUpdate update = new BatchUpdate(k2.toString());
            update.put(familyCell, String.valueOf(sum).getBytes());
            oc.collect(new ImmutableBytesWritable(familyCell.getBytes()),
update);

On Thu, Jan 27, 2011 at 6:50 PM, Stack <st...@duboce.net> wrote:

> Are there corrections up on github for v1 of the book?
>
> What you are doing looks fine.  And you have TableOutputFormat hooked
> up to the reducer?
>
> Maybe you need a close so the edits are flushed out to HBase?
>
> Can you work with 0.90.0 rather than 0.20.6?
>
> St.Ack
>
> On Thu, Jan 27, 2011 at 3:52 PM, Mark Kerzner <ma...@gmail.com>
> wrote:
> > Hi,
> >
> > this is how I write to a 0.20 HBase
> >
> >        public void reduce(Text k2, Iterator<IntWritable> itrtr,
> > OutputCollector<ImmutableBytesWritable, BatchUpdate> oc, Reporter rprtr)
> > throws IOException {
> >            int sum = 0;
> >            while (itrtr.hasNext()) {
> >                IntWritable val = itrtr.next();
> >                sum += val.get();
> >            }
> >            String family = "stats:";
> >            String familyCell = family + k2.toString();
> >            oc.collect(new ImmutableBytesWritable(familyCell.getBytes()),
> > new BatchUpdate("" + sum));
> >        }
> >
> > and the function is called, I step through it and see the correct values,
> > but then I see no rows in the output table. No error messages either.
> >
> > Could you please help?
> >
> > Thank you,
> > Mark
> >
>

Re: Writing to HBase in Reduce

Posted by Stack <st...@duboce.net>.

Are there corrections up on github for v1 of the book?

What you are doing looks fine.  And you have TableOutputFormat hooked
up to the reducer?

Maybe you need a close so the edits are flushed out to HBase?

Can you work with 0.90.0 rather than 0.20.6?

St.Ack

On Thu, Jan 27, 2011 at 3:52 PM, Mark Kerzner <ma...@gmail.com> wrote:
> Hi,
>
> this is how I write to a 0.20 HBase
>
>        public void reduce(Text k2, Iterator<IntWritable> itrtr,
> OutputCollector<ImmutableBytesWritable, BatchUpdate> oc, Reporter rprtr)
> throws IOException {
>            int sum = 0;
>            while (itrtr.hasNext()) {
>                IntWritable val = itrtr.next();
>                sum += val.get();
>            }
>            String family = "stats:";
>            String familyCell = family + k2.toString();
>            oc.collect(new ImmutableBytesWritable(familyCell.getBytes()),
> new BatchUpdate("" + sum));
>        }
>
> and the function is called, I step through it and see the correct values,
> but then I see no rows in the output table. No error messages either.
>
> Could you please help?
>
> Thank you,
> Mark
>