You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "cadl (Jira)" <ji...@apache.org> on 2022/09/16 08:25:00 UTC
[jira] [Updated] (HUDI-4858) HoodieSparkSqlWriter can't work concurrently
[ https://issues.apache.org/jira/browse/HUDI-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
cadl updated HUDI-4858:
-----------------------
Description:
When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I got a NPE at [HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609], the tableExists is ture, but tableConfig is null. The path of the table is empty, actually. Variable `tableExists` is overwritten by other write request.
We may need to remove the variable tableExists from object class properties
was:
When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I got a NPE at [HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609], the tableExists is ture, but tableConfig is null. The path of the table is empty, actually. Variable `tableExists` is overwritten by other write request.
We might remove the variable tableExists from object class properties
> HoodieSparkSqlWriter can't work concurrently
> --------------------------------------------
>
> Key: HUDI-4858
> URL: https://issues.apache.org/jira/browse/HUDI-4858
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: cadl
> Priority: Major
>
> When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I got a NPE at [HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609], the tableExists is ture, but tableConfig is null. The path of the table is empty, actually. Variable `tableExists` is overwritten by other write request.
>
> We may need to remove the variable tableExists from object class properties
--
This message was sent by Atlassian Jira
(v8.20.10#820010)