[jira] [Commented] (ACCUMULO-1454) Need good way to perform a rolling restart of all tablet servers

["Mike Drob (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13673749#comment-13673749 ] 

Mike Drob commented on ACCUMULO-1454:
-------------------------------------

bq. Maybe the 'restarting' status has a lease? Server must become responsive within a configurable time period.

+1

bq. Currently when a tablet is closed, it interrupts running scans. Waiting for scans to finish is tricky, because it seems like you would not want to allow new scans to start. So while the scans running before close would see no delay, scans started after close will still see a delay.

Would it make sense to "double host" tablets? Let existing scans finish on a tserver that is about to go down, meanwhile, load those same tablets elsewhere and point new scans to the new locations. At the end of the whole process, run one final balance to shake things out.


                
> Need good way to perform a rolling restart of all tablet servers
> ----------------------------------------------------------------
>
>                 Key: ACCUMULO-1454
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1454
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>    Affects Versions: 1.4.3, 1.5.0
>            Reporter: Mike Drob
>
> When needing to change a tserver parameter (e.g. java heap space) across the entire cluster, there is not a graceful way to perform a rolling restart.
> The naive approach of just killing tservers one at a time causes a lot of churn on the cluster as tablets move around and zookeeper tries to maintain current state.
> Potential solutions might be via a fancy fate operation, with coordination by the master. Ideally, the master would know which servers are 'safe' to restart and could minimize overall impact during the operation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira