[jira] Updated: (DERBY-801) Allow parallel access to data files.

["Anders Morken (JIRA)" <de...@db.apache.org>]

     [ http://issues.apache.org/jira/browse/DERBY-801?page=all ]

Anders Morken updated DERBY-801:
--------------------------------

    Attachment: DERBY-801-v4.patch

Thanks again for your review and comments, Knut Anders, I appreciate it. Sorry for the long delay, but I've gotten busy with my autumn project - incidentally it's related to your work on DERBY-1704. =)

Now, you've convinced me regarding updatePageArray, I've removed it from the patch. =)

Regarding writePage() while the container is in the committedDrop state, I think it can happen in two cases: Somebody's writing to the container without obeying the lock protocols in Derby - or the cache is cleaning out dirty pages it is holding to the container.

According to  BaseDataFileFactory#dropContainer(), the Container is locked in container exclusive mode while it is being closed and dropped. I haven't investigated the cache to see if it checks any such locks, but I presume it relies on it's clients to have the appropriate locks before writing to the container through the cache. 

Since the original check for the commited drop state just makes writes to a committed dropped container a noop, I figure we can do that if somebody races in and closes the container between the first check and the actual write as well - so I've added a little exception handling which hopefully does the Right Thing. Comments or better suggestions welcome, of course.

So, here we go, DERBY-801-v4.patch. Hit it. Both storeall and encryptionAll passed before I added the exception handling mentioned above, and storeall passed just great on the attached patch as well. =)

> Allow parallel access to data files.
> ------------------------------------
>
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.1.1, 10.1.1.2, 10.1.2.0, 10.1.2.1
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>         Attachments: DERBY-801-v2.patch, DERBY-801-v3.patch, DERBY-801-v4.patch, NIO-RAFContainer-v1.patch
>
>
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira