[jira] [Created] (TRAFODION-2272) UPDATE STATS on 5.5 billion row UNALIGNED table fails with HBase error 706, Retries Exhausted

["David Wayne Birdsall (JIRA)" <ji...@apache.org>]

David Wayne Birdsall created TRAFODION-2272:
-----------------------------------------------

             Summary: UPDATE STATS on 5.5 billion row UNALIGNED table fails with HBase error 706, Retries Exhausted
                 Key: TRAFODION-2272
                 URL: https://issues.apache.org/jira/browse/TRAFODION-2272
             Project: Apache Trafodion
          Issue Type: Bug
          Components: sql-cmp
    Affects Versions: 2.1-incubating
         Environment: Cluster, CDH 5.4 distribution
            Reporter: David Wayne Birdsall
            Assignee: David Wayne Birdsall


Symptom:

update statistics for table trafodion.javabench.oe_orderline_18432 on every column, (OL_W_ID, OL_I_ID), (OL_D_ID, OL_W_ID), (OL_D_ID, OL_I_ID) sample; command fails with the following:
** ERROR[9200] UPDATE STATISTICS for table TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from statement HSCursor::fetchRowset(). [2016-09-20 01:15:13]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause: 
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from statement READ_COLS_INTO_MEM. [2016-09-20 01:15:13]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause: 
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]

The table in question was unaligned format, having 5.5 billion rows. The DDL for the table is below:

CREATE TABLE TRAFODION.JAVABENCH.OE_ORDERLINE_18432
  (
    OL_O_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_D_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_I_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_SUPPLY_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
      SERIALIZED
  , OL_DELIVERY_D TIMESTAMP(6) DEFAULT NULL NOT SERIALIZED
  , OL_QUANTITY NUMERIC(2, 0) NO DEFAULT NOT NULL NOT
      DROPPABLE SERIALIZED
  , OL_AMOUNT NUMERIC(6, 2) NO DEFAULT NOT NULL NOT
      DROPPABLE SERIALIZED
  , OL_DIST_INFO CHAR(24) CHARACTER SET ISO88591 COLLATE
      DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED
  , PRIMARY KEY (OL_W_ID ASC, OL_D_ID ASC, OL_O_ID ASC, OL_NUMBER ASC)
  )
  SALT USING 64 PARTITIONS
       ON (OL_W_ID)
;

The cluster in question had 8 nodes. Memory was not an issue during the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)