[jira] [Created] (FLINK-14955) Not able to write to swift via StreamingFileSink.forBulkFormat

["Simya Jose (Jira)" <ji...@apache.org>]

Simya Jose created FLINK-14955:
----------------------------------

             Summary: Not able to write to swift via StreamingFileSink.forBulkFormat
                 Key: FLINK-14955
                 URL: https://issues.apache.org/jira/browse/FLINK-14955
             Project: Flink
          Issue Type: Bug
            Reporter: Simya Jose


not able to use StreamingFileSink to write to swift file storage

 

*Code*:

flink version: 1.9.1.

 scala 2.11

build tool : maven

main part of the code:

val eligibleItems: DataStream[EligibleItem] = env.fromCollection(Seq(
 EligibleItem("pencil"),
 EligibleItem("rubber"),
 EligibleItem("beer")))(TypeInformation.of(classOf[EligibleItem]))

val factory2: ParquetWriterFactory[EligibleItem] = ParquetAvroWriters.forReflectRecord(classOf[EligibleItem])
val sink: StreamingFileSink[EligibleItem] = StreamingFileSink
 .forBulkFormat(new Path(capHadoopPath),factory2)
 .build()

eligibleItems.addSink(sink)
 .setParallelism(1)
 .uid("TEST_1")
 .name("TEST")

*scenario* : when path is set to point to swift ( capHadoopPath = "swift://<path>" ) , getting exception - _java.lang.UnsupportedOperationException: Recoverable writers on Hadoop are only supported for HDFS and for Hadoop version 2.7 or newerjava.lang.UnsupportedOperationException: Recoverable writers on Hadoop are only supported for HDFS and for Hadoop version 2.7 or newer at org.apache.flink.fs.openstackhadoop.shaded.org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter.<init>(HadoopRecoverableWriter.java:57)_ 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)