You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Adrian Popescu <po...@yahoo.com> on 2010/01/29 21:31:23 UTC
removing incomplete indexes
Hello,
I am wondering if there is a way of removing *incomplete* indexes after I stopped the process of building them in the middle; if I try to remove it I get NullPointerException, as the index is not there, but on the other hand I cannot build the same index again, as I receive the opposite exception that this index name was already taken (TableExistsException). Is there a way to force the removal ?
Thank you,
Adrian
Re: removing incomplete indexes
Posted by Clint Morgan <cl...@troove.net>.
Is index.tableName the base table name? And index.indexName is the
indexID that you passed to createIndexedTable?
If so then looks like the metadata got corrupted. What does the
IndexedTableDescriptor look like inside of removeIndex?
(IndexedTableAdmin:111). If its gone in the meta, but the table still
exists you can simple manually remove the indexed table using the
normal mechanism.
On Sun, Jan 31, 2010 at 6:24 AM, Adrian Popescu
<po...@yahoo.com> wrote:
> Hello Stack,
>
> Yes, here is the stack trace when deleting an index from my table, the index contains the .regionfile in it but not the INDEX folder, that is why I say is incomplete. It seems that when I try to delete the index it cannot find the corresponding IndexSpecification in the IndexedTableAdmin file (line 113), and hence it crashes. On the other hand the indexed table name exists on hbase and I cannot recreate the index with the same name.
>
> See bellow and thanks !
> Adrian
>
> IndexSpecification spec = indexDesc.getIndex(indexId);
> indexDesc.removeIndex(indexId);
> here => this.disableTable(spec.getIndexedTableName(baseTableName));
>
> Stacktrace:
>
> [java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Attempting connection to server localhost/127.0.0.1:2181
> [java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/127.0.0.1:38070 remote=localhost/127.0.0.1:2181]
> [java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Server connection successful
> [java] Removing: O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index from table: orders
> [java] 10/01/31 15:03:10 INFO tableindexed.IndexedTableAdmin: Disabled orders
> [java] java.lang.NullPointerException
> [java] at org.apache.hadoop.hbase.client.tableindexed.IndexedTableAdmin.removeIndex(IndexedTableAdmin.java:113)
> [java] at hbase.query.builder.IndexGenerator.remove(Unknown Source)
> [java] at hbase.query.builder.IndexGenerator.deleteIndexes(Unknown Source)
> [java] at hbase.query.builder.IndexGenerator.main(Unknown Source)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [java] at java.lang.reflect.Method.invoke(Method.java:597)
> [java] at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
> [java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
> [java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:764)
> [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:218)
> [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:132)
> [java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:105)
> [java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [java] at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
> [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [java] at java.lang.reflect.Method.invoke(Method.java:597)
> [java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> [java] at org.apache.tools.ant.Task.perform(Task.java:348)
> [java] at org.apache.tools.ant.Target.execute(Target.java:357)
> [java] at org.apache.tools.ant.Target.performTasks(Target.java:385)
> [java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
> [java] at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
> [java] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
> [java] at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
> [java] at org.apache.tools.ant.Main.runBuild(Main.java:758)
> [java] at org.apache.tools.ant.Main.startAnt(Main.java:217)
> [java] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
> [java] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
>
>
>
>
> Where my remove function is:
> private static void remove(IndexedTableAdmin admin, Index index) {
> try {
> admin.removeIndex(Bytes.toBytes(index.tableName), index.indexName);
> }
> catch (Exception e) {
> e.printStackTrace();
> }
> }
>
>
> If I try to make another index with the same name I get this:
>
> [java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Attempting connection to server localhost/127.0.0.1:2181
> [java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/127.0.0.1:42141 remote=localhost/127.0.0.1:2181]
> [java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Server connection successful
> [java] 10/01/31 14:49:30 WARN tableindexed.IndexedTableAdmin: Adding index to existing table [orders], this may take a long time
> [java] 10/01/31 14:49:30 WARN tableindexed.IndexedTableAdmin: Not putting table in readonly, if its being written to, the index may get out of sync
> [java] org.apache.hadoop.hbase.TableExistsException: org.apache.hadoop.hbase.TableExistsException: orders-O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index
> [java] at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:793)
> [java] at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:757)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [java] at java.lang.reflect.Method.invoke(Method.java:597)
> [java] at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> [java] at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> [java]
> [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> [java] at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94)
> [java] at org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:208)
> [java] at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:165)
> [java] at org.apache.hadoop.hbase.client.tableindexed.IndexedTableAdmin.addIndex(IndexedTableAdmin.java:125)
> [java] at hbase.query.builder.IndexGenerator.createIndex(Unknown Source)
> [java] at hbase.query.builder.IndexGenerator.createIndexes(Unknown Source)
> [java] at hbase.query.builder.IndexGenerator.main(Unknown Source)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [java] at java.lang.reflect.Method.invoke(Method.java:597)
> [java] at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
> [java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
> [java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:764)
> [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:218)
> [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:132)
> [java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:105)
> [java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> [java] at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
> [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [java] at java.lang.reflect.Method.invoke(Method.java:597)
> [java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> [java] at org.apache.tools.ant.Task.perform(Task.java:348)
> [java] at org.apache.tools.ant.Target.execute(Target.java:357)
> [java] at org.apache.tools.ant.Target.performTasks(Target.java:385)
> [java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
> [java] at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
> [java] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
> [java] at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
> [java] at org.apache.tools.ant.Main.runBuild(Main.java:758)
> [java] at org.apache.tools.ant.Main.startAnt(Main.java:217)
> [java] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
> [java] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
>
>
> Moreover, if I try to remove the indexed table manually from HBase I get this:
>
> NativeException: org.apache.hadoop.hbase.client.RegionOfflineException: region offline: orders-O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index,,1264637081006
> from org/apache/hadoop/hbase/client/HConnectionManager.java:658:in `locateRegionInMeta'
> from org/apache/hadoop/hbase/client/HConnectionManager.java:594:in `locateRegion'
> from org/apache/hadoop/hbase/client/HConnectionManager.java:557:in `locateRegion'
> from org/apache/hadoop/hbase/client/HTable.java:127:in `<init>'
> from org/apache/hadoop/hbase/client/HTable.java:105:in `<init>'
>
>
>
>
> ________________________________
> From: Stack <st...@duboce.net>
> To: hbase-user@hadoop.apache.org
> Sent: Sat, January 30, 2010 9:38:22 PM
> Subject: Re: removing incomplete indexes
>
> Sorry, can you say more? Which indices and whats the full stacktrace
> on the NPE?
> Thanks,
> St.Ack
>
> On Fri, Jan 29, 2010 at 12:31 PM, Adrian Popescu
> <po...@yahoo.com> wrote:
>> Hello,
>>
>> I am wondering if there is a way of removing *incomplete* indexes after I stopped the process of building them in the middle; if I try to remove it I get NullPointerException, as the index is not there, but on the other hand I cannot build the same index again, as I receive the opposite exception that this index name was already taken (TableExistsException). Is there a way to force the removal ?
>>
>> Thank you,
>> Adrian
>>
>>
>>
>
>
>
>
Re: removing incomplete indexes
Posted by Adrian Popescu <po...@yahoo.com>.
Hello Stack,
Yes, here is the stack trace when deleting an index from my table, the index contains the .regionfile in it but not the INDEX folder, that is why I say is incomplete. It seems that when I try to delete the index it cannot find the corresponding IndexSpecification in the IndexedTableAdmin file (line 113), and hence it crashes. On the other hand the indexed table name exists on hbase and I cannot recreate the index with the same name.
See bellow and thanks !
Adrian
IndexSpecification spec = indexDesc.getIndex(indexId);
indexDesc.removeIndex(indexId);
here => this.disableTable(spec.getIndexedTableName(baseTableName));
Stacktrace:
[java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Attempting connection to server localhost/127.0.0.1:2181
[java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/127.0.0.1:38070 remote=localhost/127.0.0.1:2181]
[java] 10/01/31 15:03:08 INFO zookeeper.ClientCnxn: Server connection successful
[java] Removing: O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index from table: orders
[java] 10/01/31 15:03:10 INFO tableindexed.IndexedTableAdmin: Disabled orders
[java] java.lang.NullPointerException
[java] at org.apache.hadoop.hbase.client.tableindexed.IndexedTableAdmin.removeIndex(IndexedTableAdmin.java:113)
[java] at hbase.query.builder.IndexGenerator.remove(Unknown Source)
[java] at hbase.query.builder.IndexGenerator.deleteIndexes(Unknown Source)
[java] at hbase.query.builder.IndexGenerator.main(Unknown Source)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[java] at java.lang.reflect.Method.invoke(Method.java:597)
[java] at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
[java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
[java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:764)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:218)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:132)
[java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:105)
[java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
[java] at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[java] at java.lang.reflect.Method.invoke(Method.java:597)
[java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[java] at org.apache.tools.ant.Task.perform(Task.java:348)
[java] at org.apache.tools.ant.Target.execute(Target.java:357)
[java] at org.apache.tools.ant.Target.performTasks(Target.java:385)
[java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
[java] at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
[java] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
[java] at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
[java] at org.apache.tools.ant.Main.runBuild(Main.java:758)
[java] at org.apache.tools.ant.Main.startAnt(Main.java:217)
[java] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
[java] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
Where my remove function is:
private static void remove(IndexedTableAdmin admin, Index index) {
try {
admin.removeIndex(Bytes.toBytes(index.tableName), index.indexName);
}
catch (Exception e) {
e.printStackTrace();
}
}
If I try to make another index with the same name I get this:
[java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Attempting connection to server localhost/127.0.0.1:2181
[java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/127.0.0.1:42141 remote=localhost/127.0.0.1:2181]
[java] 10/01/31 14:49:30 INFO zookeeper.ClientCnxn: Server connection successful
[java] 10/01/31 14:49:30 WARN tableindexed.IndexedTableAdmin: Adding index to existing table [orders], this may take a long time
[java] 10/01/31 14:49:30 WARN tableindexed.IndexedTableAdmin: Not putting table in readonly, if its being written to, the index may get out of sync
[java] org.apache.hadoop.hbase.TableExistsException: org.apache.hadoop.hbase.TableExistsException: orders-O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index
[java] at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:793)
[java] at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:757)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[java] at java.lang.reflect.Method.invoke(Method.java:597)
[java] at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
[java] at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
[java]
[java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
[java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
[java] at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
[java] at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94)
[java] at org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:208)
[java] at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:165)
[java] at org.apache.hadoop.hbase.client.tableindexed.IndexedTableAdmin.addIndex(IndexedTableAdmin.java:125)
[java] at hbase.query.builder.IndexGenerator.createIndex(Unknown Source)
[java] at hbase.query.builder.IndexGenerator.createIndexes(Unknown Source)
[java] at hbase.query.builder.IndexGenerator.main(Unknown Source)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[java] at java.lang.reflect.Method.invoke(Method.java:597)
[java] at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
[java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
[java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:764)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:218)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:132)
[java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:105)
[java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
[java] at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[java] at java.lang.reflect.Method.invoke(Method.java:597)
[java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[java] at org.apache.tools.ant.Task.perform(Task.java:348)
[java] at org.apache.tools.ant.Target.execute(Target.java:357)
[java] at org.apache.tools.ant.Target.performTasks(Target.java:385)
[java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
[java] at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
[java] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
[java] at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
[java] at org.apache.tools.ant.Main.runBuild(Main.java:758)
[java] at org.apache.tools.ant.Main.startAnt(Main.java:217)
[java] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
[java] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
Moreover, if I try to remove the indexed table manually from HBase I get this:
NativeException: org.apache.hadoop.hbase.client.RegionOfflineException: region offline: orders-O_ORDERDATE_O_ORDERPRIORITY_O_ORDERKEY_index,,1264637081006
from org/apache/hadoop/hbase/client/HConnectionManager.java:658:in `locateRegionInMeta'
from org/apache/hadoop/hbase/client/HConnectionManager.java:594:in `locateRegion'
from org/apache/hadoop/hbase/client/HConnectionManager.java:557:in `locateRegion'
from org/apache/hadoop/hbase/client/HTable.java:127:in `<init>'
from org/apache/hadoop/hbase/client/HTable.java:105:in `<init>'
________________________________
From: Stack <st...@duboce.net>
To: hbase-user@hadoop.apache.org
Sent: Sat, January 30, 2010 9:38:22 PM
Subject: Re: removing incomplete indexes
Sorry, can you say more? Which indices and whats the full stacktrace
on the NPE?
Thanks,
St.Ack
On Fri, Jan 29, 2010 at 12:31 PM, Adrian Popescu
<po...@yahoo.com> wrote:
> Hello,
>
> I am wondering if there is a way of removing *incomplete* indexes after I stopped the process of building them in the middle; if I try to remove it I get NullPointerException, as the index is not there, but on the other hand I cannot build the same index again, as I receive the opposite exception that this index name was already taken (TableExistsException). Is there a way to force the removal ?
>
> Thank you,
> Adrian
>
>
>
Re: removing incomplete indexes
Posted by Stack <st...@duboce.net>.
Sorry, can you say more? Which indices and whats the full stacktrace
on the NPE?
Thanks,
St.Ack
On Fri, Jan 29, 2010 at 12:31 PM, Adrian Popescu
<po...@yahoo.com> wrote:
> Hello,
>
> I am wondering if there is a way of removing *incomplete* indexes after I stopped the process of building them in the middle; if I try to remove it I get NullPointerException, as the index is not there, but on the other hand I cannot build the same index again, as I receive the opposite exception that this index name was already taken (TableExistsException). Is there a way to force the removal ?
>
> Thank you,
> Adrian
>
>
>