[jira] [Created] (FLINK-18265) Hidden files should be ignored when the filesystem table searches for partitions

["Jingsong Lee (Jira)" <ji...@apache.org>]

Jingsong Lee created FLINK-18265:
------------------------------------

             Summary: Hidden files should be ignored when the filesystem table searches for partitions
                 Key: FLINK-18265
                 URL: https://issues.apache.org/jira/browse/FLINK-18265
             Project: Flink
          Issue Type: Bug
          Components: Connectors / FileSystem
            Reporter: Jingsong Lee
            Assignee: Jingsong Lee
             Fix For: 1.11.0


If there are some hidden files in the path of filesystem partitioned table, query this table will occur:
{code:java}
Caused by: org.apache.flink.table.api.TableException: Partition keys are: [j], incomplete partition spec: {}
        at org.apache.flink.table.filesystem.FileSystemTableSource.toFullLinkedPartSpec(FileSystemTableSource.java:209) ~[flink-table-blink_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
        at org.apache.flink.table.filesystem.FileSystemTableSource.access$800(FileSystemTableSource.java:62) ~[flink-table-blink_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
        at org.apache.flink.table.filesystem.FileSystemTableSource$1.lambda$getPaths$0(FileSystemTableSource.java:174) ~[flink-table-blink_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_152]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_152]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_152]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_152]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545) ~[?:1.8.0_152]
        at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:1.8.0_152]
        at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438) ~[?:1.8.0_152]
        at org.apache.flink.table.filesystem.FileSystemTableSource$1.getPaths(FileSystemTableSource.java:177) ~[flink-table-blink_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
{code}
Hidden files should be ignored when the filesystem table searches for partitions. This is not correct partition.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)