You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "Suresh Subbiah (JIRA)" <ji...@apache.org> on 2017/10/25 14:41:00 UTC
[jira] [Reopened] (TRAFODION-2775) Insert does not raise duplicate
row error for hbase format table with defaulted first column
[ https://issues.apache.org/jira/browse/TRAFODION-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Subbiah reopened TRAFODION-2775:
---------------------------------------
Jenkins regressions failed once after this PR was merged.During "initialize trafodion;" we get this exception stack from Java.
org.apache.hadoop.hbase.client.transactional.TransactionalTable.checkAndPut(TransactionalTable.java:795)
org.apache.hadoop.hbase.client.transactional.RMInterface.checkAndPut(RMInterface.java:635)
org.trafodion.sql.HTableClient.putRow(HTableClient.java:1594)
org.trafodion.sql.HBaseClient.insertRow(HBaseClient.java:1931).
2017-10-24 09:27:48,863, ERROR, SQL.EXE, Node Number: 0, CPU: 0, PIN: 15679, Process Name: $Z000CSZ, SQLCODE: 8448, QID: MXID11000015679212375597091582815000000000206U3333300_36_36, *** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::checkAndInsertRow returned error HBASE_ACCESS_ERROR(-706). Cause: java.io.IOException: java.lang.NullPointerException
org.apache.hadoop.hbase.client.transactional.TransactionalTable.checkAndPut(TransactionalTable.java:795)
org.apache.hadoop.hbase.client.transactional.RMInterface.checkAndPut(RMInterface.java:635)
org.trafodion.sql.HTableClient.putRow(HTableClient.java:1594)
org.trafodion.sql.HBaseClient.insertRow(HBaseClient.java:1931).
This snippet from HMaster log indicates that all MD tables were created in HBase.
2017-10-24 09:25:22,256 INFO org.apache.hadoop.hbase.master.HMaster: Client=trafodion//172.31.3.24 create 'TRAFODION._MD_.VIEWS', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|', coprocessor$3 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'}, {NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
2017-10-24 09:25:22,272 INFO org.apache.hadoop.hbase.master.handler.CreateTableHandler: Create table TRAFODION._MD_.VIEWS
2017-10-24 09:25:22,304 INFO org.apache.hadoop.hbase.regionserver.HRegion: creating HRegion TRAFODION._MD_.VIEWS HTD == 'TRAFODION._MD_.VIEWS', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|', coprocessor$3 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'}, {NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'} RootDir = hdfs://slave-cm54.trafodion.org:8020/hbase/.tmp Table name == TRAFODION._MD_.VIEWS
2017-10-24 09:25:22,315 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed TRAFODION._MD_.VIEWS,,1508837122254.fc024211ee0b77221c0b9baa891a9128.
2017-10-24 09:25:22,322 INFO org.apache.hadoop.hbase.MetaTableAccessor: Added 2
2017-10-24 09:25:22,322 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning 1 region(s) to slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:25:22,328 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {fc024211ee0b77221c0b9baa891a9128 state=OFFLINE, ts=1508837122322, server=null} to {fc024211ee0b77221c0b9baa891a9128 state=PENDING_OPEN, ts=1508837122328, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:22,334 WARN org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table TRAFODION._MD_.VIEWS state from ENABLING to ENABLED
2017-10-24 09:25:22,338 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {fc024211ee0b77221c0b9baa891a9128 state=PENDING_OPEN, ts=1508837122328, server=slave-cm54.trafodion.org,60020,1508836904162} to {fc024211ee0b77221c0b9baa891a9128 state=OPENING, ts=1508837122338, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:22,345 INFO org.apache.hadoop.hbase.master.handler.CreateTableHandler: Table, TRAFODION._MD_.VIEWS, creation successful
2017-10-24 09:25:22,374 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {fc024211ee0b77221c0b9baa891a9128 state=OPENING, ts=1508837122338, server=slave-cm54.trafodion.org,60020,1508836904162} to {fc024211ee0b77221c0b9baa891a9128 state=OPEN, ts=1508837122374, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,442 INFO org.apache.hadoop.hbase.master.HMaster: Client=trafodion//172.31.3.24 create 'TRAFODION._MD_.VIEWS_USAGE', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|', coprocessor$3 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'}, {NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
2017-10-24 09:25:23,457 INFO org.apache.hadoop.hbase.master.handler.CreateTableHandler: Create table TRAFODION._MD_.VIEWS_USAGE
2017-10-24 09:25:23,475 INFO org.apache.hadoop.hbase.regionserver.HRegion: creating HRegion TRAFODION._MD_.VIEWS_USAGE HTD == 'TRAFODION._MD_.VIEWS_USAGE', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|', coprocessor$3 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'}, {NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'} RootDir = hdfs://slave-cm54.trafodion.org:8020/hbase/.tmp Table name == TRAFODION._MD_.VIEWS_USAGE
2017-10-24 09:25:23,486 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed TRAFODION._MD_.VIEWS_USAGE,,1508837123440.12418588039df9e0e710ed90f71c3327.
2017-10-24 09:25:23,492 INFO org.apache.hadoop.hbase.MetaTableAccessor: Added 2
2017-10-24 09:25:23,492 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning 1 region(s) to slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:25:23,498 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {12418588039df9e0e710ed90f71c3327 state=OFFLINE, ts=1508837123492, server=null} to {12418588039df9e0e710ed90f71c3327 state=PENDING_OPEN, ts=1508837123498, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,504 WARN org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table TRAFODION._MD_.VIEWS_USAGE state from ENABLING to ENABLED
2017-10-24 09:25:23,524 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {12418588039df9e0e710ed90f71c3327 state=PENDING_OPEN, ts=1508837123498, server=slave-cm54.trafodion.org,60020,1508836904162} to {12418588039df9e0e710ed90f71c3327 state=OPENING, ts=1508837123524, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,530 INFO org.apache.hadoop.hbase.master.handler.CreateTableHandler: Table, TRAFODION._MD_.VIEWS_USAGE, creation successful
2017-10-24 09:25:23,555 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {12418588039df9e0e710ed90f71c3327 state=OPENING, ts=1508837123524, server=slave-cm54.trafodion.org,60020,1508836904162} to {12418588039df9e0e710ed90f71c3327 state=OPEN, ts=1508837123555, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:26:44,694 INFO org.apache.hadoop.hbase.io.hfile.LruBlockCache: totalSize=209.16 KB, freeSize=197.77 MB, max=197.98 MB, blockCount=0, accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0,evictions=29, evicted=0, evictedPerRun=0.0
2017-10-24 09:27:22,626 INFO org.apache.hadoop.hbase.master.HMaster: Client=trafodion//172.31.3.24 disable TRAFODION._MD_.AUTHS
2017-10-24 09:27:22,637 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attempting to disable table TRAFODION._MD_.AUTHS
2017-10-24 09:27:22,637 WARN org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table TRAFODION._MD_.AUTHS state from DISABLING to DISABLING
2017-10-24 09:27:22,639 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 1 regions.
2017-10-24 09:27:22,641 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {5c7b2edc80d207443f7c80976b951ea8 state=OPEN, ts=1508837099144, server=slave-cm54.trafodion.org,60020,1508836904162} to {5c7b2edc80d207443f7c80976b951ea8 state=PENDING_CLOSE, ts=1508837242641, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:27:22,665 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {5c7b2edc80d207443f7c80976b951ea8 state=PENDING_CLOSE, ts=1508837242641, server=slave-cm54.trafodion.org,60020,1508836904162} to {5c7b2edc80d207443f7c80976b951ea8 state=OFFLINE, ts=1508837242665, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:27:22,665 INFO org.apache.hadoop.hbase.master.RegionStates: Offlined 5c7b2edc80d207443f7c80976b951ea8 from slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:27:23,639 WARN org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table TRAFODION._MD_.AUTHS state from DISABLING to DISABLED
2017-10-24 09:27:23,642 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table, TRAFODION._MD_.AUTHS, is done=true
2017-10-24 09:27:23,794 INFO org.apache.hadoop.hbase.master.HMaster: Client=trafodion//172.31.3.24 delete TRAFODION._MD_.AUTHS
2017-10-24 09:27:23,806 INFO org.apache.hadoop.hbase.master.handler.TableEventHandler: Handling table operation C_M_DELETE_TABLE on table TRAFODION._MD_.AUTHS
2017-10-24 09:27:23,819 INFO org.apache.hadoop.hbase.MetaTableAccessor: Deleted [{ENCODED => 5c7b2edc80d207443f7c80976b951ea8, NAME => 'TRAFODION._MD_.AUTHS,,1508837098871.5c7b2edc80d207443f7c80976b951ea8.', STARTKEY => '', ENDKEY => ''}]
> Insert does not raise duplicate row error for hbase format table with defaulted first column
> ---------------------------------------------------------------------------------------------
>
> Key: TRAFODION-2775
> URL: https://issues.apache.org/jira/browse/TRAFODION-2775
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-exe
> Affects Versions: any
> Reporter: Suresh Subbiah
> Assignee: Suresh Subbiah
> Fix For: 2.3-incubating
>
>
> This issue was found by Gunnar Tapper and Carol Pearson.
> With HBase format Trafodion tables (each column in a row is a separate Cell), if the first column in the table can be defaulted, then uniqueness violations are not raised as expected for INSERT statements. Here is an example
> create table def1 (c1 int, c2 int not null, c3 int, primary key (c2)) attributes hbase format;
> insert into def1 (c2) values (1);
> -- raises unique constraint error as expected
> insert into def1 (c2, c3) values (1,3);
> -- does not raise constraint error
> insert into def1 (c1, c2) values (1,1);
> The problem is that during the checkAndPut for INSERTcall we are specifying the column to be checked as the one that has index 0 in the row being inserted. This would the first column being inserted into for the row, as specified by DDL, once omitted columns are removed. Columns with default value could be omitted in a given INSERT, if they are not part of the clustering key.
> The fix utilizes that fact that clustering key columns are always present in the row being inserted, even if they can be defaulted and not explicitly in the INSERT statement. We now pass in the index of the first clustering key column, in the row being inserted, to the java layer. The java layer will get the column name/qualifier from the java byte buffer version of row being inserted and use it in the CheckAndPut call. Not that the index of first clustering key column will depend on both which default columns are being skipped and order of columns in DDL. This index does not depend only on DDL.
> With the change we get the expected error
> >>insert into def1 (c1, c2) values (1,1);
> *** ERROR[8102] The operation is prevented by a unique constraint.
> --- 0 row(s) inserted.
> >>insert into def1 (c2, c1) values (51,1);
> --- 1 row(s) inserted.
> >>insert into def1 (c2, c1) values (1,51);
> *** ERROR[8102] The operation is prevented by a unique constraint.
> --- 0 row(s) inserted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)