You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by "angers.zhu" <an...@gmail.com> on 2019/12/18 12:35:02 UTC

[VOTE][SPARK-29018][SPIP]:Build spark thrift server based on protocol v11

With the development of Spark and Hive,in current sql/hive-thriftserver
module,

we need to do a lot of work to solve code conflicts for different built-in
hive versions.

It's an annoying and unending work in current ways. And these issues have
limited

our ability and convenience to develop new features for Spark’s thrift server.

    We suppose to implement a new thrift server and JDBC driver based on Hive’s latest v11 

TCLService.thrift thrift protocol. Finally, the new thrift server have below
feature:

  1. Build new module spark-service as spark’s thrift server 

  2. Don't need as much reflection and inherited code as `hive-thriftser` modules

  3. Support all functions current `sql/hive-thriftserver` support

  4. Use all code maintained by spark itself, won’t depend on Hive

  5. Support origin functions use spark’s own way, won't limited by Hive's code

  6. Support running without hive metastore or with hive metastore

  7. Support user impersonation by Multi-tenant splited hive authentication and DFS authentication

  8. Support session hook for with spark’s own code

  9. Add a new jdbc driver spark-jdbc, with spark’s own connection url  “jdbc:spark:<host>:<port>/<db>”

  10. Support both hive-jdbc and spark-jdbc client, then we can support most clients and BI platform

  

<https://issues.apache.org/jira/browse/SPARK-29018>



Google Doc:
<https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#>

  
[ ] +1: Accept the proposal as an official SPIP  
[ ] +0  
[ ] -1: I don't think this is a good idea because ...  
  
I'll start with my +1

[ ![](https://mail-online.nosdn.127.net/qiyelogo/defaultAvatar.png) |
angers.zhu  
---|---  
angers.zhu@gmail.com  
](https://maas.mail.163.com/dashi-web-
extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-
online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D)

签名由 [网易邮箱大师](https://mail.163.com/dashi/dlpro.html?from=mail81) 定制