You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "Alice Chen (JIRA)" <ji...@apache.org> on 2015/07/22 20:18:02 UTC

[jira] [Created] (TRAFODION-749) LP Bug: 1391271 - Random update statistics failures with hbase OutOfOrderScannerNextException

Alice Chen created TRAFODION-749:
------------------------------------

             Summary: LP Bug: 1391271 - Random update statistics failures with hbase OutOfOrderScannerNextException
                 Key: TRAFODION-749
                 URL: https://issues.apache.org/jira/browse/TRAFODION-749
             Project: Apache Trafodion
          Issue Type: Bug
          Components: sql-cmp
            Reporter: Weishiun Tsai
            Assignee: Barry Fritchman
            Priority: Critical
             Fix For: 2.0-incubating


During the QA table population with the v1029_0830 build and the v1107_0830 build on a Cloudera cluster, update statistics encountered this error 4 times.  This OutOfOrderScannerNextException has not shown up for quite some time.  Some recent changes in update statics may have caused the error to return again.  This bug report is to document this problem, so that some research can be done in this area to figure out what in update statistics may have stressed hbase to return this error again.

Here is the 1 seen from the v1029_0830 build:

SQL>update statistics for table cbase on every column sample random 5 percent;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_WISC32.CBASE encountered an error (8448) from statement Process_Query. [2014-10-31 09:34:00]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-10-31 09:34:00]

Here are the 3 seen from the v1107_0830 build:

SQL>update statistics for table trafodion.g_tpcds1x.inventory on every column sample;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_TPCDS1X.INVENTORY encountered an error (8448) from statement Process_Query. [2014-11-08 13:01:55]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 13:01:55]

SQL>update statistics for table trafodion.g_tpcds1x.store_sales on every column sample;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_TPCDS1X.STORE_SALES encountered an error (8448) from statement Process_Query. [2014-11-08 13:14:29]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 13:14:29]

SQL>update statistics for table cbase on every column sample random 5 percent;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_WISC32.CBASE encountered an error (8448) from statement Process_Query. [2014-11-08 16:15:06]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 16:15:06]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)