[jira] [Comment Edited] (DRILL-4286) Have an ability to put server in quiescent mode of operation

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323484#comment-15323484 ] 

Paul Rogers edited comment on DRILL-4286 at 6/9/16 10:22 PM:
-------------------------------------------------------------

Details of ZK revisions:

drill
. drillbits
. . <protobuf encoded node id>
. registry
. . <node id, perhaps host:user port>
. . . state
. . . http port (not advertised in ZK today)
. . . (other metadata to be added later)…

The Drillbit would create the /drill/drillbits entry first, then populate the registry data. Older or (dumber) clients can listen to only /drill/drillbits today and will work as today (though they will try to send queries to a quiescent node and have those queries rejected.) Also, if the registry znodes are in simple text format, then a varity of clients can read the data without need of Protobuf decoders.

State would be one of:

(missing) - Drillbit is initializing, ignore this Drillbit for now
START - Initializing (still ignore the Drillbit)
RUN - Normal state
DRAIN - Queries complete, no new queries, please.
STOP - Transient state during shut-down

The semantics for a listener are: first notice node in the drillbits znode as today. For newer clients & servers, then check the registry znode and watch the state znode. Only send queries to Drillbits in the RUN state. Live queries are fine in the RUN or DRAIN state. Panic if the Drillbit disappears or transitions to the STOP state.

The Drillbit itself accepts queries only while in the RUN state.

We then need a way to trigger the graceful shutdown. I would prefer a simple REST message (with authentication) to trigger shut-down. This is the technique we use with the Drill-on-YARN application master: it requires no fancy client code, no protobuf, no ZK.

The existing SIGTERM technique is still needed to keep the drillbit.sh stop operation simple. Think of SIGTERM as the local solution, the REST API /rest/stop message as the remote solution.

No work has been started on this; I present this here to see if anyone has suggestions for a simpler solution.


was (Author: paul-rogers):
Details of ZK revisions:

/drill
|- drillbits
  |- [protobuf encoded node id]
|- registry
  |- <node id, perhaps host:user port>
    |- state
    |- http port (not advertised in ZK today)
    |- (other metadata to be added later)…

The Drillbit would create the /drill/drillbits entry first, then populate the registry data. Older or (dumber) clients can listen to only /drill/drillbits today and will work as today (though they will try to send queries to a quiescent node and have those queries rejected.)

State would be one of:

(missing) - Drillbit is initializing, ignore this Drillbit for now
START - Initializing (still ignore the Drillbit)
RUN - Normal state
DRAIN - Queries complete, no new queries, please.
STOP - Transient state during shut-down

The semantics for a listener are: first notice node in the drillbits znode as today. For newer clients & servers, then check the registry znode and watch the state znode. Only send queries to Drillbits in the RUN state. Live queries are fine in the RUN or DRAIN state. Panic if the Drillbit disappears or transitions to the STOP state.

The Drillbit itself accepts queries only while in the RUN state.

We then need a way to trigger the graceful shutdown. I would prefer a simple REST message (with authentication) to trigger shut-down. This is the technique we use with the Drill-on-YARN application master: it requires no fancy client code, no protobuf, no ZK.

The existing SIGTERM technique is still needed to keep the drillbit.sh stop operation simple. Think of SIGTERM as the local solution, the REST API /rest/stop message as the remote solution.

No work has been started on this; I present this here to see if anyone has suggestions for a simpler solution.

> Have an ability to put server in quiescent mode of operation
> ------------------------------------------------------------
>
>                 Key: DRILL-4286
>                 URL: https://issues.apache.org/jira/browse/DRILL-4286
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Execution - Flow
>            Reporter: Victoria Markman
>
> I think drill will benefit from mode of operation that is called "quiescent" in some databases. 
> From IBM Informix server documentation:
> {code}
> Change gracefully from online to quiescent mode
> Take the database server gracefully from online mode to quiescent mode to restrict access to the database server without interrupting current processing. After you perform this task, the database server sets a flag that prevents new sessions from gaining access to the database server. The current sessions are allowed to finish processing. After you initiate the mode change, it cannot be canceled. During the mode change from online to quiescent, the database server is considered to be in Shutdown mode.
> {code}
> This is different from shutdown, when processes are terminated. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)