[jira] [Created] (ACCUMULO-2438) deleting all table rows caused tablet load failure

["Eric Newton (JIRA)" <ji...@apache.org>]

Eric Newton created ACCUMULO-2438:
-------------------------------------

             Summary: deleting all table rows caused tablet load failure
                 Key: ACCUMULO-2438
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2438
             Project: Accumulo
          Issue Type: Bug
          Components: master
         Environment: nightly single-node test
            Reporter: Eric Newton
            Assignee: Eric Newton
            Priority: Critical
             Fix For: 1.6.0


testTrace in ConditionalWriterIT failed:

{noformat}
java.lang.Exception: test timed out after 60000 milliseconds
	at java.lang.Thread.sleep(Native Method)
	at org.apache.accumulo.core.util.UtilWaitThread.sleep(UtilWaitThread.java:26)
	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:240)
	at org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:84)
	at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:177)
	at org.apache.accumulo.core.trace.TraceDump.printTrace(TraceDump.java:133)
	at org.apache.accumulo.test.ConditionalWriterIT.testTrace(ConditionalWriterIT.java:1231)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}

Examining the logs, it seems like the client was having a hard time locating the trace table (table id 9):

{noformat}
2014-03-07 04:33:40,445 [impl.ThriftScanner] TRACE: Failed to locate tablet for table : 9 row : 9dd07d1bd8b9bc0d
...
2014-03-07 04:33:40,551 [impl.ThriftScanner] TRACE: Failed to locate tablet for table : 9 row : 9dd07d1bd8b9bc0d
{noformat}

Examining the master logs, I see the trace table being created, assigned, and hosted.

This test deletes everything in the trace table before tracing an operation:

{noformat}
    conn.tableOperations().deleteRows("trace", null, null);
{noformat}

The master takes the table offline and re-writes the metadata table information so that it no longer has any data.

The master then attempts to put the tablet online, but the operation fails:

{noformat}
2014-03-07 04:33:40,079 [master.Master] ERROR: host1:47630 reports assignment failed for tablet 9<<
{noformat}

The tablet server gives the real problem:

{noformat}
2014-03-07 04:33:40,074 [tserver.TabletServer] WARN : exception trying to assign tablet 9<< file:////local/disk1/jenkins/workspace/accumulo16/test/target/mini-tests/org.apache.accumulo.test.functional.SimpleMacIT/1394184783304_4048/accumulo/tables/9/default_tablet
java.lang.IllegalArgumentException: Time type unknown : ^@0
        at org.apache.accumulo.server.tablets.TabletTime.getInstance(TabletTime.java:67)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1296)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1211)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1067)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1056)
        at org.apache.accumulo.tserver.TabletServer$AssignmentHandler.run(TabletServer.java:2911)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at java.lang.Thread.run(Thread.java:701)
{noformat}

Tablet time should always be "L" or "M".




--
This message was sent by Atlassian JIRA
(v6.2#6252)