You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2024/06/05 00:58:00 UTC
[jira] [Closed] (HUDI-7796) Gracefully cast file system instance in Avro writers
[ https://issues.apache.org/jira/browse/HUDI-7796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Guo closed HUDI-7796.
---------------------------
Resolution: Fixed
> Gracefully cast file system instance in Avro writers
> ----------------------------------------------------
>
> Key: HUDI-7796
> URL: https://issues.apache.org/jira/browse/HUDI-7796
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.15.0, 1.0.0
>
>
> When running tests in Trino with Hudi MDT enabled, the following line in HoodieAvroHFileWriter throws class cast exception, since Trino uses dependency injection to provide the Hadoop file system instance, which may skip the Hudi wrapper file system logic.
> {code:java}
> this.fs = (HoodieWrapperFileSystem) this.file.getFileSystem(conf); {code}
> {code:java}
> Caused by: java.lang.ClassCastException: class io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper cannot be cast to class org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem (io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper and org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem are in unnamed module of loader 'app')
> at org.apache.hudi.io.hadoop.HoodieAvroHFileWriter.<init>(HoodieAvroHFileWriter.java:91)
> at org.apache.hudi.io.hadoop.HoodieAvroFileWriterFactory.newHFileFileWriter(HoodieAvroFileWriterFactory.java:108)
> at org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriterByFormat(HoodieFileWriterFactory.java:70)
> at org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriter(HoodieFileWriterFactory.java:53)
> at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:108)
> at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:77)
> at org.apache.hudi.io.CreateHandleFactory.create(CreateHandleFactory.java:45)
> at org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:101)
> at org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:44) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)