You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Fabian Hueske (JIRA)" <ji...@apache.org> on 2015/02/04 23:22:36 UTC

[jira] [Closed] (FLINK-454) Add ProgramInput/OutputFormats

     [ https://issues.apache.org/jira/browse/FLINK-454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fabian Hueske closed FLINK-454.
-------------------------------
    Resolution: Duplicate

Is subsumed by issues around FLINK-986 to cache intermediate results.

> Add ProgramInput/OutputFormats
> ------------------------------
>
>                 Key: FLINK-454
>                 URL: https://issues.apache.org/jira/browse/FLINK-454
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: GitHub Import
>            Priority: Minor
>              Labels: github-import
>             Fix For: pre-apache
>
>
> It would be nice to be able to plug existing Stratosphere programs together. 
> This eases the use of program libraries, such as for machine learning or spatial data.
> Right now a library algorithm would be used as follows:
> 1. Run a program that preprocessed data, brings it into the correct format for the library algorithm and writes it to a FS.
> 1. Run the algorithm, which reads its input from FS and write the result back.
> 1. Maybe have a postprocessing job, which reads again from FS.
> By providing ProgramInput/OutputFormats, these programs could be directly connected, allowing for: 
> 1. pipelined processing
> 1. cross program optimization 
> 1. elimination of a driver program
> 1. combination of different programming abstraction in one job (Spargel, Stratosphere Java, etc.)
> 1. ...
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/454
> Created by: [fhueske|https://github.com/fhueske]
> Labels: enhancement, user satisfaction, 
> Created at: Mon Feb 03 21:14:05 CET 2014
> State: open



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)