[jira] [Commented] (HTRACE-314) htraced: make datastore loading safer

["Colin Patrick McCabe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HTRACE-314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036700#comment-15036700 ] 

Colin Patrick McCabe commented on HTRACE-314:
---------------------------------------------

This patch cleans up and simplifies the loading process by moving it out of {{datastore.go}} and into {{loader.go}}.  The {{DataStoreLoader}} object can be used by other things besides the normal htraced startup process... unit tests can use it to poke at leveldb, for example, and we eventually can add a {{fsck}} command or similar using the {{DataStoreLoader}}.

We now have a unique {{DaemonId}} which all leveldb instances will get, that will make it clear that they belong to the same htraced (or not).  We also have a {{ShardIndex}} and {{TotalShards}} field stored in each leveldb which will allow us to know how to re-assemble the leveldbs.  Since we are hashing a specific subset of all shards to each leveldb, we don't want to assemble the datastore in the wrong order.  We can add new fields to the {{ShardInfo}} structure later and they will be treated as optional.

As an optimization, we only open the leveldb instances once... the dataStore can just take ownership of the leveldb instances which the loader opened rather than re-doing that work.

> htraced: make datastore loading safer
> -------------------------------------
>
>                 Key: HTRACE-314
>                 URL: https://issues.apache.org/jira/browse/HTRACE-314
>             Project: HTrace
>          Issue Type: Improvement
>          Components: htraced
>    Affects Versions: 4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HTRACE-314.001.patch
>
>
> We should make htraced datastore loading safer by verifying that we have the correct number of leveldb instances, assembled in the correct order, when starting up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)