[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #2585: [Feature][ConnectorV2]add file excel sink

[GitBox <gi...@apache.org>]

Hisoka-X commented on PR #2585:
URL: https://github.com/apache/incubator-seatunnel/pull/2585#issuecomment-1236679797

   > > > > ![image](https://user-images.githubusercontent.com/51053924/188068629-c98f8289-7983-4441-8334-56a11017de6b.png)
   > > > > You e2e test cases also have some problems, please check.
   > > > 
   > > > 
   > > > ![image](https://user-images.githubusercontent.com/32196893/188150962-092773d3-0ec8-4105-b8c8-699ab960cef1.png) If I add this code, Excel sink Write's `finishAndCloseWriteFile` method will be called twice when executed with Flink, resulting in an error!
   > > > However, if you remove this code and run it with Spark, Text sink Write is executed instead of Excel sink Write
   > > 
   > > 
   > > Can you provide the detailed call stack information?
   > 
   > ![image](https://user-images.githubusercontent.com/32196893/188264087-5a9c4b08-5cb1-44e8-bbd6-faba49a05f92.png) ![image](https://user-images.githubusercontent.com/32196893/188264111-9e60de8b-05c3-4145-a318-ad49df7cda99.png)
   > 
   > `SparkDataWriter's` `commit()` method clears `commitInfo` after execution, However, the `prepareCommit` of `FlinkSinkWrite` does not clear commitInfo, so it writes `ExcelWorkBook` when it calls the close() method of `FlinkSlinkWrite`, but it does the WorkBook when it executes `prepareCommit` It has been closed. I don't understand why SparkDataWriter and FlinkSinkWrite have different commit logic. Do I need to determine whether the workbook is close in `finishAndCloseWriteFile`?
   
   The reason already in comment: `combine the prepareCommit and commit in this method.`. Flink Support Committer and GlobalCommitter, but spark only support GlobalCommitter(same logic different name). So spark use commit() to run prepareCommit() and Committer.commit().


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org