[jira] [Assigned] (HUDI-3693) error will be occured when writing data into hudi using flink job more than once in same flink session

["chacha.tang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chacha.tang reassigned HUDI-3693:
---------------------------------

    Assignee: chacha.tang

> error will be occured when writing data into hudi using flink job more than once in same flink session
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-3693
>                 URL: https://issues.apache.org/jira/browse/HUDI-3693
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: flink
>            Reporter: chacha.tang
>            Assignee: chacha.tang
>            Priority: Minor
>
> 环境：
> hudi: 0.10.1
> flink:1.13.2
> flink cluster setup in standalone mode, and use the below flink sql to launch the flink job
> #create the cow sink table
> CREATE TABLE t1(
>   uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED,
>   name VARCHAR(10),
>   age INT,
>   ts TIMESTAMP(3),
>   `partition` VARCHAR(20)
> )
> PARTITIONED BY (`partition`)
> WITH (
>   'connector' = 'hudi',
>   'path' = '/user/hive/warehouse/hudi.db/t1',
>   'write.tasks' = '1',
>   'table.type' = 'COPY_ON_WRITE'
> );
> #write data first
> INSERT INTO t1 VALUES ('id1','Danny',20,TIMESTAMP '1970-01-01 00:00:01','par1')
> #write data  in twice
> INSERT INTO t1 VALUES ('id1','Danny',20,TIMESTAMP '1970-01-01 00:00:01','par1')
> the first time the flink job is finished successfully, but it failed in twice, and the below exception occured 
> org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to org.apache.hudi.common.fs.HoodieWrapperFileSystem
> I search issues, and found issue#3885 mentioned same problem, But the root casuse is different.My problem is involed by hadoop fs cache. The HoodieWrapperFileSystem is cached in fs cache when it create by first flink job in flink taskmanger.When I launched the second flink job to write data, it used cached HoodieWrapperFileSystem and whose childFlinkClassload is different with this flink job.So the exception occured.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)