You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Tianying Chang <ty...@gmail.com> on 2014/02/07 06:53:20 UTC
WALPlayer?
Hi, folks
I want to try the WALPlayer. But it complains not found. Am I running it
the wrong way?
Thanks
Tian-Ying
hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
Unknown program 'WALPlayer' chosen.
Valid program names are:
CellCounter: Count cells in HBase table
completebulkload: Complete a bulk data load.
copytable: Export a table from local cluster to peer cluster
export: Write table data to HDFS.
import: Import data written by Export.
importtsv: Import data in TSV format.
rowcounter: Count rows in HBase table
verifyrep: Compare the data from tables in two different clusters.
WARNING: It doesn't work for incrementColumnValues'd cells since the
timestamp is changed after being appended to the log.
Re: WALPlayer?
Posted by Tianying Chang <ty...@gmail.com>.
Thanks, that works. the new table has more data. I will verify the count.
Thanks
Tian-Ying
On Tue, Feb 11, 2014 at 11:42 AM, Matteo Bertozzi
<th...@gmail.com>wrote:
> I think that the problem here is that you're trying to replay the WAL for
> TestTable-clone entries..
> which are not present... you probably want to replay the entries from the
> original TestTable.
>
> I think that you can specify a mapping.. something like WalPlayer TestTable
> TestTable-cloned
>
> Matteo
>
>
>
> On Tue, Feb 11, 2014 at 7:37 PM, Tianying Chang <ty...@gmail.com> wrote:
>
> > I am trying to use snapshot+WALPlayer for HBase DR for our cluster in
> AWS.
> > I am trying to do below to verify it, seems the new data is not being
> > played into the new table. Anything wrong with my steps?
> > 1. Populate TestTable using PeformanceEvaluation Tool
> > 2. count the rows being written, 63277 row(s)
> > 3. take a snapshot, then clone a table TestTable-cloned based on this
> > snapshot. Count the row # and verified that is has same number of rows as
> > TestTable
> > 4. write more data to TestTable using PerformanceEvaluation
> > 5. count the row of TestTable, which has more rows.
> > 6. call WALPlayer: su tychang hbase
> > org.apache.hadoop.hbase.mapreduce.WALPlayer /hbase/.logs TestTable-cloned
> > 7. count the row of TestTable-cloned. Find the row count is not changed,
> > still 63277 row(s) :(
> >
> > I suspect that the WAL is not rolled yet, so WALPlayer cannot replay
> those
> > data. So I populated more data to make sure the WAL log rolled.(I can see
> > the count of hlogs increased by 10) But still, after running WALPlayer,
> the
> > rowCount is not changed. Any idea? Can WALPlayer work with Snapshot?
> >
> > Thanks
> > Tian-Ying
> >
> >
> > On Fri, Feb 7, 2014 at 10:31 AM, Tianying Chang <ty...@gmail.com>
> wrote:
> >
> > > Hi, Lars
> > >
> > > Sure. I will come back and update the thread.
> > >
> > > Thanks
> > > Tian-Ying
> > >
> > >
> > > On Fri, Feb 7, 2014 at 10:17 AM, lars hofhansl <la...@apache.org>
> wrote:
> > >
> > >> Let me know how this works for you.
> > >> I wrote that tool a while ago, but I ended up never actually using
> > myself.
> > >>
> > >> -- Lars
> > >>
> > >>
> > >>
> > >> ________________________________
> > >> From: Tianying Chang <ty...@gmail.com>
> > >> To: user@hbase.apache.org
> > >> Sent: Thursday, February 6, 2014 10:06 PM
> > >> Subject: Re: WALPlayer?
> > >>
> > >>
> > >> Never mind. Should use hbase command. :)
> > >>
> > >> Thanks
> > >> Tian-Ying
> > >>
> > >>
> > >>
> > >> On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com>
> > wrote:
> > >>
> > >> > Hi, folks
> > >> >
> > >> > I want to try the WALPlayer. But it complains not found. Am I
> running
> > it
> > >> > the wrong way?
> > >> >
> > >> > Thanks
> > >> > Tian-Ying
> > >> >
> > >> > hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
> > >> >
> > >> > Unknown program 'WALPlayer' chosen.
> > >> >
> > >> > Valid program names are:
> > >> >
> > >> > CellCounter: Count cells in HBase table
> > >> >
> > >> > completebulkload: Complete a bulk data load.
> > >> >
> > >> > copytable: Export a table from local cluster to peer cluster
> > >> >
> > >> > export: Write table data to HDFS.
> > >> >
> > >> > import: Import data written by Export.
> > >> >
> > >> > importtsv: Import data in TSV format.
> > >> >
> > >> > rowcounter: Count rows in HBase table
> > >> >
> > >> > verifyrep: Compare the data from tables in two different clusters.
> > >> > WARNING: It doesn't work for incrementColumnValues'd cells since the
> > >> > timestamp is changed after being appended to the log.
> > >> >
> > >>
> > >
> > >
> >
>
Re: WALPlayer?
Posted by Matteo Bertozzi <th...@gmail.com>.
I think that the problem here is that you're trying to replay the WAL for
TestTable-clone entries..
which are not present... you probably want to replay the entries from the
original TestTable.
I think that you can specify a mapping.. something like WalPlayer TestTable
TestTable-cloned
Matteo
On Tue, Feb 11, 2014 at 7:37 PM, Tianying Chang <ty...@gmail.com> wrote:
> I am trying to use snapshot+WALPlayer for HBase DR for our cluster in AWS.
> I am trying to do below to verify it, seems the new data is not being
> played into the new table. Anything wrong with my steps?
> 1. Populate TestTable using PeformanceEvaluation Tool
> 2. count the rows being written, 63277 row(s)
> 3. take a snapshot, then clone a table TestTable-cloned based on this
> snapshot. Count the row # and verified that is has same number of rows as
> TestTable
> 4. write more data to TestTable using PerformanceEvaluation
> 5. count the row of TestTable, which has more rows.
> 6. call WALPlayer: su tychang hbase
> org.apache.hadoop.hbase.mapreduce.WALPlayer /hbase/.logs TestTable-cloned
> 7. count the row of TestTable-cloned. Find the row count is not changed,
> still 63277 row(s) :(
>
> I suspect that the WAL is not rolled yet, so WALPlayer cannot replay those
> data. So I populated more data to make sure the WAL log rolled.(I can see
> the count of hlogs increased by 10) But still, after running WALPlayer, the
> rowCount is not changed. Any idea? Can WALPlayer work with Snapshot?
>
> Thanks
> Tian-Ying
>
>
> On Fri, Feb 7, 2014 at 10:31 AM, Tianying Chang <ty...@gmail.com> wrote:
>
> > Hi, Lars
> >
> > Sure. I will come back and update the thread.
> >
> > Thanks
> > Tian-Ying
> >
> >
> > On Fri, Feb 7, 2014 at 10:17 AM, lars hofhansl <la...@apache.org> wrote:
> >
> >> Let me know how this works for you.
> >> I wrote that tool a while ago, but I ended up never actually using
> myself.
> >>
> >> -- Lars
> >>
> >>
> >>
> >> ________________________________
> >> From: Tianying Chang <ty...@gmail.com>
> >> To: user@hbase.apache.org
> >> Sent: Thursday, February 6, 2014 10:06 PM
> >> Subject: Re: WALPlayer?
> >>
> >>
> >> Never mind. Should use hbase command. :)
> >>
> >> Thanks
> >> Tian-Ying
> >>
> >>
> >>
> >> On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com>
> wrote:
> >>
> >> > Hi, folks
> >> >
> >> > I want to try the WALPlayer. But it complains not found. Am I running
> it
> >> > the wrong way?
> >> >
> >> > Thanks
> >> > Tian-Ying
> >> >
> >> > hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
> >> >
> >> > Unknown program 'WALPlayer' chosen.
> >> >
> >> > Valid program names are:
> >> >
> >> > CellCounter: Count cells in HBase table
> >> >
> >> > completebulkload: Complete a bulk data load.
> >> >
> >> > copytable: Export a table from local cluster to peer cluster
> >> >
> >> > export: Write table data to HDFS.
> >> >
> >> > import: Import data written by Export.
> >> >
> >> > importtsv: Import data in TSV format.
> >> >
> >> > rowcounter: Count rows in HBase table
> >> >
> >> > verifyrep: Compare the data from tables in two different clusters.
> >> > WARNING: It doesn't work for incrementColumnValues'd cells since the
> >> > timestamp is changed after being appended to the log.
> >> >
> >>
> >
> >
>
Re: WALPlayer?
Posted by Tianying Chang <ty...@gmail.com>.
I am trying to use snapshot+WALPlayer for HBase DR for our cluster in AWS.
I am trying to do below to verify it, seems the new data is not being
played into the new table. Anything wrong with my steps?
1. Populate TestTable using PeformanceEvaluation Tool
2. count the rows being written, 63277 row(s)
3. take a snapshot, then clone a table TestTable-cloned based on this
snapshot. Count the row # and verified that is has same number of rows as
TestTable
4. write more data to TestTable using PerformanceEvaluation
5. count the row of TestTable, which has more rows.
6. call WALPlayer: su tychang hbase
org.apache.hadoop.hbase.mapreduce.WALPlayer /hbase/.logs TestTable-cloned
7. count the row of TestTable-cloned. Find the row count is not changed,
still 63277 row(s) :(
I suspect that the WAL is not rolled yet, so WALPlayer cannot replay those
data. So I populated more data to make sure the WAL log rolled.(I can see
the count of hlogs increased by 10) But still, after running WALPlayer, the
rowCount is not changed. Any idea? Can WALPlayer work with Snapshot?
Thanks
Tian-Ying
On Fri, Feb 7, 2014 at 10:31 AM, Tianying Chang <ty...@gmail.com> wrote:
> Hi, Lars
>
> Sure. I will come back and update the thread.
>
> Thanks
> Tian-Ying
>
>
> On Fri, Feb 7, 2014 at 10:17 AM, lars hofhansl <la...@apache.org> wrote:
>
>> Let me know how this works for you.
>> I wrote that tool a while ago, but I ended up never actually using myself.
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: Tianying Chang <ty...@gmail.com>
>> To: user@hbase.apache.org
>> Sent: Thursday, February 6, 2014 10:06 PM
>> Subject: Re: WALPlayer?
>>
>>
>> Never mind. Should use hbase command. :)
>>
>> Thanks
>> Tian-Ying
>>
>>
>>
>> On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com> wrote:
>>
>> > Hi, folks
>> >
>> > I want to try the WALPlayer. But it complains not found. Am I running it
>> > the wrong way?
>> >
>> > Thanks
>> > Tian-Ying
>> >
>> > hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
>> >
>> > Unknown program 'WALPlayer' chosen.
>> >
>> > Valid program names are:
>> >
>> > CellCounter: Count cells in HBase table
>> >
>> > completebulkload: Complete a bulk data load.
>> >
>> > copytable: Export a table from local cluster to peer cluster
>> >
>> > export: Write table data to HDFS.
>> >
>> > import: Import data written by Export.
>> >
>> > importtsv: Import data in TSV format.
>> >
>> > rowcounter: Count rows in HBase table
>> >
>> > verifyrep: Compare the data from tables in two different clusters.
>> > WARNING: It doesn't work for incrementColumnValues'd cells since the
>> > timestamp is changed after being appended to the log.
>> >
>>
>
>
Re: WALPlayer?
Posted by Tianying Chang <ty...@gmail.com>.
Hi, Lars
Sure. I will come back and update the thread.
Thanks
Tian-Ying
On Fri, Feb 7, 2014 at 10:17 AM, lars hofhansl <la...@apache.org> wrote:
> Let me know how this works for you.
> I wrote that tool a while ago, but I ended up never actually using myself.
>
> -- Lars
>
>
>
> ________________________________
> From: Tianying Chang <ty...@gmail.com>
> To: user@hbase.apache.org
> Sent: Thursday, February 6, 2014 10:06 PM
> Subject: Re: WALPlayer?
>
>
> Never mind. Should use hbase command. :)
>
> Thanks
> Tian-Ying
>
>
>
> On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com> wrote:
>
> > Hi, folks
> >
> > I want to try the WALPlayer. But it complains not found. Am I running it
> > the wrong way?
> >
> > Thanks
> > Tian-Ying
> >
> > hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
> >
> > Unknown program 'WALPlayer' chosen.
> >
> > Valid program names are:
> >
> > CellCounter: Count cells in HBase table
> >
> > completebulkload: Complete a bulk data load.
> >
> > copytable: Export a table from local cluster to peer cluster
> >
> > export: Write table data to HDFS.
> >
> > import: Import data written by Export.
> >
> > importtsv: Import data in TSV format.
> >
> > rowcounter: Count rows in HBase table
> >
> > verifyrep: Compare the data from tables in two different clusters.
> > WARNING: It doesn't work for incrementColumnValues'd cells since the
> > timestamp is changed after being appended to the log.
> >
>
Re: WALPlayer?
Posted by lars hofhansl <la...@apache.org>.
Let me know how this works for you.
I wrote that tool a while ago, but I ended up never actually using myself.
-- Lars
________________________________
From: Tianying Chang <ty...@gmail.com>
To: user@hbase.apache.org
Sent: Thursday, February 6, 2014 10:06 PM
Subject: Re: WALPlayer?
Never mind. Should use hbase command. :)
Thanks
Tian-Ying
On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com> wrote:
> Hi, folks
>
> I want to try the WALPlayer. But it complains not found. Am I running it
> the wrong way?
>
> Thanks
> Tian-Ying
>
> hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
>
> Unknown program 'WALPlayer' chosen.
>
> Valid program names are:
>
> CellCounter: Count cells in HBase table
>
> completebulkload: Complete a bulk data load.
>
> copytable: Export a table from local cluster to peer cluster
>
> export: Write table data to HDFS.
>
> import: Import data written by Export.
>
> importtsv: Import data in TSV format.
>
> rowcounter: Count rows in HBase table
>
> verifyrep: Compare the data from tables in two different clusters.
> WARNING: It doesn't work for incrementColumnValues'd cells since the
> timestamp is changed after being appended to the log.
>
Re: WALPlayer?
Posted by Tianying Chang <ty...@gmail.com>.
Never mind. Should use hbase command. :)
Thanks
Tian-Ying
On Thu, Feb 6, 2014 at 9:53 PM, Tianying Chang <ty...@gmail.com> wrote:
> Hi, folks
>
> I want to try the WALPlayer. But it complains not found. Am I running it
> the wrong way?
>
> Thanks
> Tian-Ying
>
> hadoop jar /tmp/hbase-0.94.17-SNAPSHOT.jar WALPlayer
>
> Unknown program 'WALPlayer' chosen.
>
> Valid program names are:
>
> CellCounter: Count cells in HBase table
>
> completebulkload: Complete a bulk data load.
>
> copytable: Export a table from local cluster to peer cluster
>
> export: Write table data to HDFS.
>
> import: Import data written by Export.
>
> importtsv: Import data in TSV format.
>
> rowcounter: Count rows in HBase table
>
> verifyrep: Compare the data from tables in two different clusters.
> WARNING: It doesn't work for incrementColumnValues'd cells since the
> timestamp is changed after being appended to the log.
>