[jira] [Updated] (SPARK-19978) spark thrift server to switch to normative hadoop 2.2+ service lifecycle

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-19978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-19978:
---------------------------------
    Labels: bulk-closed  (was: )

> spark thrift server to switch to normative hadoop 2.2+ service lifecycle
> ------------------------------------------------------------------------
>
>                 Key: SPARK-19978
>                 URL: https://issues.apache.org/jira/browse/SPARK-19978
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.2.0
>            Reporter: Steve Loughran
>            Priority: Minor
>              Labels: bulk-closed
>
> Spark's hive-thriftserver module (for which there isn't a JIRA component yet, BTW), contains, in {{org.apache.hive.service}}, a copy and paste of Hive's service lifecyle classes.
> Hive's classes are in fact a copy and paste of the Hadoop 0.23 service lifecycle classes. Those were reworked, "fixed" for Hadoop 2.2 by all the changes in YARN-117
> in particular
> * restructuring of subclass point means that the state checks happen at the right place to stop re-entrancy
> * no race conditions on the methods
> * resilient stop
> * composite service handles failure of child services properly
> * very handy diagnostics logging of lifecycles
> * robust listener API.
> + various other fixes
> The hive thriftserver services can and should move to the hadoop 2.2 ones, which will take a couple of hours, testing included. 
> Essentially
> # change parent
> # move subclass operations from init/start/stop to serviceInit(), serviceStart and serviceStop() methods
> # test
> The only troublespot is hive's calls take a HiveConf in init; I'd fix that by still taking a Hadoop Configuration (the superclass) but having a {[HiveService extends AbstractService}} that always makes sure the stored conf is a HiveConf, and add a getHiveConf() method for subclasses to access it. The subclasses will also get a guarantee that their {{serviceInit(Configuration)}} call will always be supplied a config which can be cast to HiveConf. I'll need to add a CompositeService.
> If people want this, I can do it. If they don't, I wont bother.
> As to why Hive have an out-of-date subclass of the original lifecycle stuff? That topic has been raise in the past, it always came down to "we wanted to run on older versions of Hadoop".  They can revisit that discussion...doesn't mean other projects can't make their own choices



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org