You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Sean Bridges (JIRA)" <ji...@apache.org> on 2018/12/04 18:08:00 UTC
[jira] [Comment Edited] (DERBY-7023) Exception inserting (possibly
due to integer overflow)
[ https://issues.apache.org/jira/browse/DERBY-7023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709047#comment-16709047 ]
Sean Bridges edited comment on DERBY-7023 at 12/4/18 6:07 PM:
--------------------------------------------------------------
table signature is,
{code:java}
create table iqQueue (
id BIGINT NOT NULL PRIMARY KEY,
msgId CHAR(16) FOR BIT DATA NOT NULL UNIQUE
);{code}
the failing statement is,
{code:java}
insert into iqQueue (id, msgId) values (?, ?){code}
we use default settings except for,
{code:java}
"derby.storage.pageCacheSize", "5000"
"derby.storage.pageSize", "16384"{code}
running with verbose logging doesn't show much more.
The table is being used as a queue, typically the table is nearly empty, but we add/remove millions of rows daily (and have been doing so for several years now).
Looking at this,
[https://db.apache.org/derby/papers/pageformats.html]
I think we have overflown the next record identifier which is only 4 bytes (though you could double capacity by treating those bytes as an unsigned long).
was (Author: sbridges):
table signature is,
{code:java}
create table iqQueue (
id BIGINT NOT NULL PRIMARY KEY,
msgId CHAR(16) FOR BIT DATA NOT NULL UNIQUE
);{code}
the failing statement is,
{code:java}
insert into iqQueuedDeletes (id, msg) values (?, ?){code}
we use default settings except for,
{code:java}
"derby.storage.pageCacheSize", "5000"
"derby.storage.pageSize", "16384"{code}
running with verbose logging doesn't show much more.
The table is being used as a queue, typically the table is nearly empty, but we add/remove millions of rows daily (and have been doing so for several years now).
Looking at this,
[https://db.apache.org/derby/papers/pageformats.html]
I think we have overflown the next record identifier which is only 4 bytes (though you could double capacity by treating those bytes as an unsigned long).
> Exception inserting (possibly due to integer overflow)
> ------------------------------------------------------
>
> Key: DERBY-7023
> URL: https://issues.apache.org/jira/browse/DERBY-7023
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.10.1.1
> Reporter: Sean Bridges
> Priority: Major
> Fix For: 10.15.0.0
>
>
> We get an exception writing to derby,
>
> {code:java}
> Caused by: java.sql.SQLException: An unexpected exception was thrown
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:101)
> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:148)
> at org.apache.derby.impl.jdbc.Util.seeNextException(Util.java:349)
> at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:431)
> at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:353)
> at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2400)
> at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:85)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1436)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1724)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPreparedStatement.java:1379)
> at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
> at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
> ... 1 more
> Caused by: java.sql.SQLException: An unexpected exception was thrown
> at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:42)
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:126)
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
> ... 14 more
> Caused by: java.sql.SQLException: Java exception: ': java.io.IOException'.
> at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:42)
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:126)
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:148)
> at org.apache.derby.impl.jdbc.Util.javaException(Util.java:370)
> at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:436)
> at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:432)
> ... 11 more
> Caused by: java.io.IOException
> at org.apache.derby.iapi.services.io.CompressedNumber.writeInt(CompressedNumber.java:97)
> at org.apache.derby.impl.store.raw.data.StoredRecordHeader.write(StoredRecordHeader.java:457)
> at org.apache.derby.impl.store.raw.data.StoredPage.logRow(StoredPage.java:3861)
> at org.apache.derby.impl.store.raw.data.InsertOperation.writeOptionalDataToBuffer(InsertOperation.java:370)
> at org.apache.derby.impl.store.raw.data.InsertOperation.<init>(InsertOperation.java:115)
> at org.apache.derby.impl.store.raw.data.LoggableActions.actionInsert(LoggableActions.java:139)
> at org.apache.derby.impl.store.raw.data.BasePage.insertNoOverflow(BasePage.java:604)
> at org.apache.derby.impl.store.raw.data.BasePage.insertAtSlot(BasePage.java:525)
> at org.apache.derby.impl.store.raw.data.StoredPage.insertAtSlot(StoredPage.java:6833)
> at org.apache.derby.impl.store.access.btree.BTreeController.doIns(BTreeController.java:938)
> at org.apache.derby.impl.store.access.btree.BTreeController.insert(BTreeController.java:1372)
> at org.apache.derby.impl.store.access.btree.index.B2IController.insert(B2IController.java:210)
> at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(IndexChanger.java:440)
> at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(IndexChanger.java:383)
> at org.apache.derby.impl.sql.execute.IndexChanger.insert(IndexChanger.java:590)
> at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(IndexSetChanger.java:268)
> at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(RowChangerImpl.java:453)
> at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1057)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:518)
> at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:452)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:333)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1344)
> ... 7 more{code}
>
> I think this is due to an integer overflow.
> Earlier StoredPage#readPageHeader is called, and nextId is set to -2147481666. Before this line in StoredPage#readPageHeader,
> {code:java}
> nextId = lrdi.nextInt();{code}
> lrdi.position = 16, and
> lrdi.pageData = (in hex) 007500000001000000017f41de880219800007be0000000000...
> I can't provide the entire data base, as it is confidential.
> This occurred with derby 10.10.1.0, but I can reproduce it with 10.14.1.0
> We do a lot of writes/deletes to this table, it's quite possible we've hit 2 billion inserts/deletes.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)