[jira] [Commented] (DERBY-5936) AllocPage.ReadContainerInfo throws ArrayIndexOutOfBoundsException at arraycopy

["Mike Matrigali (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DERBY-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13929820#comment-13929820 ] 

Mike Matrigali commented on DERBY-5936:
---------------------------------------

Just going to add some thoughts after looking at code in the stack trace.  I have
a good idea at this point what is root cause.

In the first report by Yuan  in this issue the store is failing to open a container while
doing boot time redo recovery.  From stack can not tell if it is a system table/index, or
user table/index.  Any failure like this during redo recovery will cause the database
to not boot, and likely will happen every subsequent time an attempt is made to reboot the
database as the same redo recovery code will be executed.  

In this case user reports that database was not closed "cleanly", which explains the trip
through the boot time recovery code.  

I can't tell from the stack if we are dealing with a dropped table or not.  The call to
openDroppedContainer just means that during recovery we may need to open a 
dropped container.  the interface probably should be called openPossiblyDroppedContainer.
But a subsequent user mentioned that a jvm crash likely happened during a compress, which
causes a number of dropped containers, and I know we have had problems in the past, it 
is a good area to think about.

A dropped container can either come from an actual user level sql drop, or there are a number
of ddl statements that derby will internally generate drops for.  The most usual case is
SYSCS_UTIL.SYSCS_COMPRESS_TABLE(), which internally copies all the data from each
user table and user index container into a new container and drops the old containers, then
manipulates the system catalogs to fix up the pointers.

dropped containers can go through a few transitions in derby, depending on recovery needs.  
First the entire container can just be renamed from a cXXX.dat file to dXXX.dat file.  After derby
does not need the data in the old file for recovery, but still may need to open the file during recovery 
a "stub" file is created with the same header so open works, but no data is left.  I would guess that
this is the area causing this problem - but it is just a guess.  There are multiple comments in the
code recognizing that this is a tricky place, but still expect AllocPage.ReadContainerInfo to work.
For instance in AllocPage.ReadContainerInfo:
@param epage the allocation page which has the container information.
        Epage may not be fully formed, but is guarenteed to be big enough to
        cover the area inhibited by the container info

And in its caller, FileContainer.readHeader:
        When this method is called, the embryonic page that is passed in must
        have been read directly from the file or the input stream, even if the
        alloc page may still be in cache.  This is because a stubbify operation
        only writes the stub to disk, it does not get rid of any stale page
        from the page cache.  So if it so happens that the stubbified container
        object is aged out of the container cache but the first alloc page
        hasn't, then when any stale page of this container wants to be written
        out, the container needs to be reopened, which is when this routine is
        called.  We must not get the alloc page in cache because it may be
        stale page and it may still say the container has not been dropped.



> AllocPage.ReadContainerInfo throws ArrayIndexOutOfBoundsException at arraycopy
> ------------------------------------------------------------------------------
>
>                 Key: DERBY-5936
>                 URL: https://issues.apache.org/jira/browse/DERBY-5936
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.6.1.0
>         Environment: intel, windows XP, embedded driver, c3p0
>            Reporter: Yuan Yao
>
> The db may not be closed normally. It throws following exceptions as starting.
> logs:
> {code}
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> 	at java.lang.System.arraycopy(Native Method)
> 	at org.apache.derby.impl.store.raw.data.AllocPage.ReadContainerInfo(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.FileContainer.readHeader(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.openContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.RAFContainer4.openContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.FileContainer.setIdent(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.setIdentity(Unknown Source)
> 	at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openDroppedContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.xact.Xact.openDroppedContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.findContainer(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.needsRedo(Unknown Source)
> 	at org.apache.derby.impl.store.raw.log.FileLogger.redo(Unknown Source)
> 	at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
> 	at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
> 	at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
> 	at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
> 	... 24 more
> {code}
> I'd like to upload data files if necessary.



--
This message was sent by Atlassian JIRA
(v6.2#6252)