You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Guangle Fan <fa...@gmail.com> on 2014/09/10 20:13:44 UTC
copy snapshot for scanner "no table descriptor found"
Hi,
I'm running hbase 0.98.1
When using TableMapReduceUtil to init a MR job on a snapshot, I got an
error :
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file
under hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:481)
at
org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:725)
at
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.setInput(TableSnapshotInputFormat.java:394)
at
org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableSnapshotMapperJob(TableMapReduceUtil.java:300)
at com.flipboard.etl.MRJobs2.aggregateMagazineMetaByUser(MRJobs2.java:538)
at com.flipboard.etl.MapReduceRunner2.run(MapReduceRunner2.java:164)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at com.flipboard.etl.MapReduceRunner2.main(MapReduceRunner2.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
It's looking for .tableinfo file under snapshot directory, but there are
only two files there :
-rwxr-xr-x 3 hbase hadoop 51 2014-09-10 00:49
hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/.snapshotinfo
-rw-r--r-- 3 hbase hadoop 11733 2014-09-10 00:49
hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/data.manifest
The snapshot was created by hbase shell :
snapshot 'xxx', 'xxx-snapshot'
Any ideas why there is such inconsistency ?
Thanks!
Guangle
Re: copy snapshot for scanner "no table descriptor found"
Posted by Matteo Bertozzi <th...@gmail.com>.
correct.. you can't use new stuff from old versions
Matteo
On Wed, Sep 10, 2014 at 12:41 PM, Guangle Fan <fa...@gmail.com> wrote:
> Sorry correct my words.
> The snapshot was created by 0.98.1 cdh5.1.0 (new format of snapshot)
> And reading where the failure happened is using 0.96.1.1 cdh5.0.1 (still
> looking for old format)
>
> On Wed, Sep 10, 2014 at 12:35 PM, Guangle Fan <fa...@gmail.com>
> wrote:
>
> > Hi, Matteo
> >
> > Thanks for the replay! The version I'm running is 0.98.1 cdh5.1.0 from
> > Cloudera, Yes, it doesn't apply that new format yet.
> >
> > Guangle
> >
> > On Wed, Sep 10, 2014 at 11:50 AM, Matteo Bertozzi <
> theo.bertozzi@gmail.com
> > > wrote:
> >
> >> you are probably not using hbase 0.98.1 or you were mixing different
> >> version
> >> The data.manifest got in master/0.98.6 so, the snapshot is generated
> with
> >> one of those,
> >> and you are may be reading with a version that does not support the new
> >> format
> >>
> >> Matteo
> >>
> >>
> >> On Wed, Sep 10, 2014 at 11:13 AM, Guangle Fan <fa...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm running hbase 0.98.1
> >> >
> >> > When using TableMapReduceUtil to init a MR job on a snapshot, I got an
> >> > error :
> >> >
> >> > org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor
> >> file
> >> > under hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot
> >> >
> >> > at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:481)
> >> >
> >> > at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:725)
> >> >
> >> > at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.setInput(TableSnapshotInputFormat.java:394)
> >> >
> >> > at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableSnapshotMapperJob(TableMapReduceUtil.java:300)
> >> >
> >> > at
> >> com.flipboard.etl.MRJobs2.aggregateMagazineMetaByUser(MRJobs2.java:538)
> >> >
> >> > at com.flipboard.etl.MapReduceRunner2.run(MapReduceRunner2.java:164)
> >> >
> >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> >
> >> > at com.flipboard.etl.MapReduceRunner2.main(MapReduceRunner2.java:68)
> >> >
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >
> >> > at
> >> >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> >
> >> > at
> >> >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >
> >> > at java.lang.reflect.Method.invoke(Method.java:597)
> >> >
> >> > at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> >> >
> >> >
> >> > It's looking for .tableinfo file under snapshot directory, but there
> are
> >> > only two files there :
> >> >
> >> > -rwxr-xr-x 3 hbase hadoop 51 2014-09-10 00:49
> >> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/.snapshotinfo
> >> >
> >> > -rw-r--r-- 3 hbase hadoop 11733 2014-09-10 00:49
> >> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/data.manifest
> >> >
> >> >
> >> > The snapshot was created by hbase shell :
> >> >
> >> > snapshot 'xxx', 'xxx-snapshot'
> >> >
> >> >
> >> > Any ideas why there is such inconsistency ?
> >> >
> >> >
> >> > Thanks!
> >> >
> >> >
> >> > Guangle
> >> >
> >>
> >
> >
>
Re: copy snapshot for scanner "no table descriptor found"
Posted by Guangle Fan <fa...@gmail.com>.
Sorry correct my words.
The snapshot was created by 0.98.1 cdh5.1.0 (new format of snapshot)
And reading where the failure happened is using 0.96.1.1 cdh5.0.1 (still
looking for old format)
On Wed, Sep 10, 2014 at 12:35 PM, Guangle Fan <fa...@gmail.com> wrote:
> Hi, Matteo
>
> Thanks for the replay! The version I'm running is 0.98.1 cdh5.1.0 from
> Cloudera, Yes, it doesn't apply that new format yet.
>
> Guangle
>
> On Wed, Sep 10, 2014 at 11:50 AM, Matteo Bertozzi <theo.bertozzi@gmail.com
> > wrote:
>
>> you are probably not using hbase 0.98.1 or you were mixing different
>> version
>> The data.manifest got in master/0.98.6 so, the snapshot is generated with
>> one of those,
>> and you are may be reading with a version that does not support the new
>> format
>>
>> Matteo
>>
>>
>> On Wed, Sep 10, 2014 at 11:13 AM, Guangle Fan <fa...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I'm running hbase 0.98.1
>> >
>> > When using TableMapReduceUtil to init a MR job on a snapshot, I got an
>> > error :
>> >
>> > org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor
>> file
>> > under hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot
>> >
>> > at
>> >
>> >
>> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:481)
>> >
>> > at
>> >
>> >
>> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:725)
>> >
>> > at
>> >
>> >
>> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.setInput(TableSnapshotInputFormat.java:394)
>> >
>> > at
>> >
>> >
>> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableSnapshotMapperJob(TableMapReduceUtil.java:300)
>> >
>> > at
>> com.flipboard.etl.MRJobs2.aggregateMagazineMetaByUser(MRJobs2.java:538)
>> >
>> > at com.flipboard.etl.MapReduceRunner2.run(MapReduceRunner2.java:164)
>> >
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> >
>> > at com.flipboard.etl.MapReduceRunner2.main(MapReduceRunner2.java:68)
>> >
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >
>> > at
>> >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >
>> > at
>> >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> >
>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> >
>> >
>> > It's looking for .tableinfo file under snapshot directory, but there are
>> > only two files there :
>> >
>> > -rwxr-xr-x 3 hbase hadoop 51 2014-09-10 00:49
>> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/.snapshotinfo
>> >
>> > -rw-r--r-- 3 hbase hadoop 11733 2014-09-10 00:49
>> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/data.manifest
>> >
>> >
>> > The snapshot was created by hbase shell :
>> >
>> > snapshot 'xxx', 'xxx-snapshot'
>> >
>> >
>> > Any ideas why there is such inconsistency ?
>> >
>> >
>> > Thanks!
>> >
>> >
>> > Guangle
>> >
>>
>
>
Re: copy snapshot for scanner "no table descriptor found"
Posted by Guangle Fan <fa...@gmail.com>.
Hi, Matteo
Thanks for the replay! The version I'm running is 0.98.1 cdh5.1.0 from
Cloudera, Yes, it doesn't apply that new format yet.
Guangle
On Wed, Sep 10, 2014 at 11:50 AM, Matteo Bertozzi <th...@gmail.com>
wrote:
> you are probably not using hbase 0.98.1 or you were mixing different
> version
> The data.manifest got in master/0.98.6 so, the snapshot is generated with
> one of those,
> and you are may be reading with a version that does not support the new
> format
>
> Matteo
>
>
> On Wed, Sep 10, 2014 at 11:13 AM, Guangle Fan <fa...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm running hbase 0.98.1
> >
> > When using TableMapReduceUtil to init a MR job on a snapshot, I got an
> > error :
> >
> > org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor
> file
> > under hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot
> >
> > at
> >
> >
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:481)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:725)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.setInput(TableSnapshotInputFormat.java:394)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableSnapshotMapperJob(TableMapReduceUtil.java:300)
> >
> > at
> com.flipboard.etl.MRJobs2.aggregateMagazineMetaByUser(MRJobs2.java:538)
> >
> > at com.flipboard.etl.MapReduceRunner2.run(MapReduceRunner2.java:164)
> >
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >
> > at com.flipboard.etl.MapReduceRunner2.main(MapReduceRunner2.java:68)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> >
> >
> > It's looking for .tableinfo file under snapshot directory, but there are
> > only two files there :
> >
> > -rwxr-xr-x 3 hbase hadoop 51 2014-09-10 00:49
> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/.snapshotinfo
> >
> > -rw-r--r-- 3 hbase hadoop 11733 2014-09-10 00:49
> > hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/data.manifest
> >
> >
> > The snapshot was created by hbase shell :
> >
> > snapshot 'xxx', 'xxx-snapshot'
> >
> >
> > Any ideas why there is such inconsistency ?
> >
> >
> > Thanks!
> >
> >
> > Guangle
> >
>
Re: copy snapshot for scanner "no table descriptor found"
Posted by Matteo Bertozzi <th...@gmail.com>.
you are probably not using hbase 0.98.1 or you were mixing different version
The data.manifest got in master/0.98.6 so, the snapshot is generated with
one of those,
and you are may be reading with a version that does not support the new
format
Matteo
On Wed, Sep 10, 2014 at 11:13 AM, Guangle Fan <fa...@gmail.com> wrote:
> Hi,
>
> I'm running hbase 0.98.1
>
> When using TableMapReduceUtil to init a MR job on a snapshot, I got an
> error :
>
> org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file
> under hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot
>
> at
>
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:481)
>
> at
>
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:725)
>
> at
>
> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.setInput(TableSnapshotInputFormat.java:394)
>
> at
>
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableSnapshotMapperJob(TableMapReduceUtil.java:300)
>
> at com.flipboard.etl.MRJobs2.aggregateMagazineMetaByUser(MRJobs2.java:538)
>
> at com.flipboard.etl.MapReduceRunner2.run(MapReduceRunner2.java:164)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
> at com.flipboard.etl.MapReduceRunner2.main(MapReduceRunner2.java:68)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
>
> It's looking for .tableinfo file under snapshot directory, but there are
> only two files there :
>
> -rwxr-xr-x 3 hbase hadoop 51 2014-09-10 00:49
> hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/.snapshotinfo
>
> -rw-r--r-- 3 hbase hadoop 11733 2014-09-10 00:49
> hdfs://hbase/hbase/.hbase-snapshot/xxx-snapshot/data.manifest
>
>
> The snapshot was created by hbase shell :
>
> snapshot 'xxx', 'xxx-snapshot'
>
>
> Any ideas why there is such inconsistency ?
>
>
> Thanks!
>
>
> Guangle
>