[jira] [Updated] (SOLR-2700) transaction logging

["Yonik Seeley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yonik Seeley updated SOLR-2700:
-------------------------------

    Attachment: SOLR-2700.patch

Here's a draft patch.
There is a tlog.<number> file created for each commit.  The javabin format is used to serialize SolrInputDocuments.
An in-memory map of pointers into the log is kept for documents not yet soft-committed, and the realtime-get component checks that first before using SolrCore.getNewestSearcher().

Seems to work for getting documents not in the newest searcher so far.

Tons of stuff left to do
- the tlog files are currently in the CWD
- need to handle deletes
- need to handle flushes in a performant way
- need to implement optional fsync for durability on power-failure
- would be nice to make some of this multi-threaded for better performance
- need to implement durability (apply updates from logs on startup)
- need to implement some form of cleanup for transaction logs


> transaction logging
> -------------------
>
>                 Key: SOLR-2700
>                 URL: https://issues.apache.org/jira/browse/SOLR-2700
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>         Attachments: SOLR-2700.patch
>
>
> A transaction log is needed for durability of updates, for a more performant realtime-get, and for replaying updates to recovering peers.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org