You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Pei He (Jira)" <ji...@apache.org> on 2020/09/06 20:33:00 UTC
[jira] [Created] (FLINK-19144) Error when writing to partitioned
table with s3 FileSystem
Pei He created FLINK-19144:
------------------------------
Summary: Error when writing to partitioned table with s3 FileSystem
Key: FLINK-19144
URL: https://issues.apache.org/jira/browse/FLINK-19144
Project: Flink
Issue Type: Bug
Components: Connectors / Hive
Affects Versions: 1.11.1
Reporter: Pei He
It looks like HadoopFileSystemFactory is created in [here|[https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableSink.java#L134].] However, it cannot recognize s3 files system implementations which are based on org.apache.flink.core.fs.FileSystemFactory.
{code:java}
Caused by: java.io.IOException: No FileSystem for scheme: s3Caused by: java.io.IOException: No FileSystem for scheme: s3 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389) ~[hadoop-common-2.8.3.jar:?] at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356) ~[hadoop-common-2.8.3.jar:?] at org.apache.flink.connectors.hive.HadoopFileSystemFactory.create(HadoopFileSystemFactory.java:46) ~[flink-connector-hive_2.11-1.11.1.jar:1.11.1] at org.apache.flink.table.filesystem.stream.StreamingFileCommitter.lambda$initializeState$0(Hive.java:125) ~[flink-table-blink_2.11-1.11.1.jar:1.11.1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)