You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/06/24 15:03:24 UTC
[jira] [Commented] (FLINK-838) GSoC Summer Project: Implement full
Hadoop Compatibility Layer for Stratosphere
[ https://issues.apache.org/jira/browse/FLINK-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042071#comment-14042071 ]
ASF GitHub Bot commented on FLINK-838:
--------------------------------------
GitHub user atsikiridis opened a pull request:
https://github.com/apache/incubator-flink/pull/37
hadoopcompatibility: Implementations of basic programming interfaces and...
... a basic driver.
* wrappers for Mapper, Reducer and Combiner (as a local Reducer) on the new Java API
* wrapper for OutputCollector
* wrappers for Partitioner, values comparator.
* a driver making it posible to run unmonitored (so far) Hadoop jobs on Flink.
* tests and variations of the WordCount driver exclusively in Hadoop.
You can find more about the project here: https://issues.apache.org/jira/browse/FLINK-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031511
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/atsikiridis/incubator-flink gsoc-midterm
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/37.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #37
----
commit 6aaccc71910074c3130228a7855d25cfcfce3e82
Author: Artem Tsikiridis <ar...@cern.ch>
Date: 2014-05-13T19:31:13Z
hadoopcompatibility: Implementations of basic programming interfaces and a basic driver.
* wrappers for Mapper, Reducer and Combiner (as a local Reducer) on the new Java API
* wrapper for OutputCollector
* wrappers for Partitioner, values comparator.
* a driver making it posible to run unmonitored (so far) Hadoop jobs on Flink.
* tests and variations of the WordCount driver exclusively in Hadoop.
----
> GSoC Summer Project: Implement full Hadoop Compatibility Layer for Stratosphere
> -------------------------------------------------------------------------------
>
> Key: FLINK-838
> URL: https://issues.apache.org/jira/browse/FLINK-838
> Project: Flink
> Issue Type: Improvement
> Reporter: GitHub Import
> Labels: github-import
> Fix For: pre-apache
>
>
> This is a meta issue for tracking @atsikiridis progress with implementing a full Hadoop Compatibliltiy Layer for Stratosphere.
> Some documentation can be found in the Wiki: https://github.com/stratosphere/stratosphere/wiki/%5BGSoC-14%5D-A-Hadoop-abstraction-layer-for-Stratosphere-(Project-Map-and-Notes)
> As well as the project proposal: https://github.com/stratosphere/stratosphere/wiki/GSoC-2014-Project-Proposal-Draft-by-Artem-Tsikiridis
> Most importantly, there is the following **schedule**:
> *19 May - 27 June (Midterm)*
> 1) Work on the Hadoop tasks, their Context and the mapping of Hadoop's Configuration to the one of Stratosphere. By successfully bridging the Hadoop tasks with Stratosphere, we already cover the most basic Hadoop Jobs. This can be determined by running some popular Hadoop examples on Stratosphere (e.g. WordCount, k-means, join) (4 - 5 weeks)
> 2) Understand how the running of these jobs works (e.g. command line interface) for the wrapper. Implement how will the user run them. (1 - 2 weeks).
> *27 June - 11 August*
> 1) Continue wrapping more "advanced" Hadoop Interfaces (Comparators, Partitioners, Distributed Cache etc.) There are quite a few interfaces and it will be a challenge to support all of them. (5 full weeks)
> 2) Profiling of the application and optimizations (if applicable)
> *11 August - 18 August*
> Write documentation on code, write a README with care and add more unit-tests. (1 week)
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/838
> Created by: [rmetzger|https://github.com/rmetzger]
> Labels: core, enhancement, parent-for-major-feature,
> Milestone: Release 0.7 (unplanned)
> Created at: Tue May 20 10:11:34 CEST 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.2#6252)