You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by spoganshev <s....@slice.com> on 2019/05/27 13:30:30 UTC
Re: FileInputFormat that processes files in chronological order
Why is FileInputFormat#getInputSplitAssigner not configurable though? It
makes sense to let those who use FileInputFormat set the desired split
assigner (and make LocatableInputSplitAssigner just a default one).
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: FileInputFormat that processes files in chronological order
Posted by Fabian Hueske <fh...@gmail.com>.
I see, that's unfortunate.
Both classes are also tagged with @Public, making them unchangeable until
Flink 2.0.
Nonetheless, feel free to open a Jira issue to improve the situation for a
future release.
Best, Fabian
Am Mo., 27. Mai 2019 um 16:55 Uhr schrieb spoganshev <s.poganshev@slice.com
>:
> I've tried that, but the problem is:
> - FileInputFormat#getInputSplitAssigner return type is
> LocatableInputSplitAssigner
> - LocatableInputSplitAssigner is final
>
> Which makes it impossible to override the split assigner unfortunately
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
Re: FileInputFormat that processes files in chronological order
Posted by spoganshev <s....@slice.com>.
I've tried that, but the problem is:
- FileInputFormat#getInputSplitAssigner return type is
LocatableInputSplitAssigner
- LocatableInputSplitAssigner is final
Which makes it impossible to override the split assigner unfortunately
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: FileInputFormat that processes files in chronological order
Posted by Fabian Hueske <fh...@gmail.com>.
Configuring the split assigner wasn't a common requirement so far.
You can just implement your own format extending from FileInputFormat (or
any of its subclasses) and override the getInputSplitAssigner() method.
Best, Fabian
Am Mo., 27. Mai 2019 um 15:30 Uhr schrieb spoganshev <s.poganshev@slice.com
>:
> Why is FileInputFormat#getInputSplitAssigner not configurable though? It
> makes sense to let those who use FileInputFormat set the desired split
> assigner (and make LocatableInputSplitAssigner just a default one).
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>