You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/12/15 17:52:13 UTC
[jira] [Commented] (FLINK-1236) Add support for localization of
Hadoop Input Splits
[ https://issues.apache.org/jira/browse/FLINK-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246823#comment-14246823 ]
ASF GitHub Bot commented on FLINK-1236:
---------------------------------------
GitHub user fhueske opened a pull request:
https://github.com/apache/incubator-flink/pull/267
[FLINK-1236] Add local split assignment support for HadoopInputFormats
Forwards the location information from Hadoop InputSplits to Flink InputSplits to enable local split assignment.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/incubator-flink localHadoopSplits
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/267.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #267
----
commit dc728b10205f162b1ea89ea42000d5609b446e76
Author: Fabian Hueske <fh...@apache.org>
Date: 2014-12-15T14:18:27Z
[FLINK-1236] Add local split assignment support for HadoopInputFormats
----
> Add support for localization of Hadoop Input Splits
> ---------------------------------------------------
>
> Key: FLINK-1236
> URL: https://issues.apache.org/jira/browse/FLINK-1236
> Project: Flink
> Issue Type: Improvement
> Reporter: Robert Metzger
> Assignee: Fabian Hueske
> Priority: Minor
> Labels: starter
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Flink is currently not localizing splits created by Hadoop Input formats.
> The getInputSplitAssigner() of the HadoopInputFormat is returning the DefaultInputSplitAssigner instead of the LocatableInputSplitAssigner().
> It should be possible to make the HadoopInputSplit a LocatableInputSplit, because Hadoop's InputSplit also contains the hostnames, for exactly this purpose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)