You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "XiaoyuGeng (Jira)" <ji...@apache.org> on 2022/07/18 13:21:00 UTC

[jira] [Created] (HUDI-4415) Support spark writer running on thrift server

XiaoyuGeng created HUDI-4415:
--------------------------------

             Summary: Support spark writer running on thrift server
                 Key: HUDI-4415
                 URL: https://issues.apache.org/jira/browse/HUDI-4415
             Project: Apache Hudi
          Issue Type: New Feature
          Components: spark-sql
            Reporter: XiaoyuGeng


Spark thrift server manages a sql by running with the thread local spark session. It identifies the jobs belong to the sql by setting the job group Id. If user wants to cancel the sql running, thrift server will call session.cancelJobGroup. However, the implementation of hudi spark writer uses different job group Ids to distinguish the different proccesing part of the writing progress, like workprofile, compaction, rollback and so on. There is an Incompatibility



--
This message was sent by Atlassian Jira
(v8.20.10#820010)