You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andy Feng (JIRA)" <ji...@apache.org> on 2017/11/30 00:18:00 UTC
[jira] [Updated] (SPARK-22658) SPIP: TeansorFlowOnSpark as a
Scalable Deep Learning Lib of Apache Spark
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Feng updated SPARK-22658:
------------------------------
Description:
In Feburary 2017, TensorFlowOnSpark (TFoS) was released for distributed TensorFlow training and inference on Apache Spark clusters. TFoS is designed to:
* Easily migrate all existing TensorFlow programs with minimum code change;
* Support all TensorFlow functionalities: synchronous/asynchronous training, model/data parallelism, inference and TensorBoard;
* Easily integrate with your existing data processing pipelines (ex. Spark SQL) and machine learning algorithms (ex. MLlib);
* Be easily deployed on cloud or on-premise: CPU & GPU, Ethernet and Infiniband.
We propose to merge TFoS into Apache Spark as a scalable deep learning library to:
* Make deep learning easy for Apache Spark community: Familiar pipeline API for training and inference; Enable TensorFlow training/inference on existing Spark clusters.
* Further simplify data scientist experience: Ensure compatibility b/w Apache Spark and TFoS; Reduce steps for installation.
* Help Apache Spark evolution on deep learning: Establish a design pattern for additional frameworks (ex. Caffe, CNTK); Structured streaming for DL training/inference.
was:
In Feburary 2017, TensorFlowOnSpark (TFoS) was released for distributed TensorFlow training and inference on Apache Spark clusters. TFoS is designed to:
* Easily migrate all existing TensorFlow programs with minimum code change;
* Support all TensorFlow functionalities: synchronous/asynchronous training, model/data parallelism, inference and TensorBoard;
* Easily integrate with your existing data processing pipelines (ex. Spark SQL) and machine learning algorithms (ex. MLlib);
* Be easily deployed on cloud or on-premise: CPU & GPU, Ethernet and Infiniband.
We propose to merge TFoS into Apache Spark as a scalable deep learning library to:
* Make deep learning easy for Apache Spark community: Familiar pipeline API for training and inference; Enable TensorFlow training/inference on existing Spark clusters.
* Further simplify data scientist experience: Ensure compatibility b/w Apache Spark and TFoS;
Reduce steps for installation.
* Help Apache Spark evolution on deep learning: Establish a design pattern for additional frameworks (ex. Caffe, CNTK); Structured streaming for DL training/inference.
> SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark
> ------------------------------------------------------------------------
>
> Key: SPARK-22658
> URL: https://issues.apache.org/jira/browse/SPARK-22658
> Project: Spark
> Issue Type: New Feature
> Components: ML
> Affects Versions: 2.2.0
> Reporter: Andy Feng
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> In Feburary 2017, TensorFlowOnSpark (TFoS) was released for distributed TensorFlow training and inference on Apache Spark clusters. TFoS is designed to:
> * Easily migrate all existing TensorFlow programs with minimum code change;
> * Support all TensorFlow functionalities: synchronous/asynchronous training, model/data parallelism, inference and TensorBoard;
> * Easily integrate with your existing data processing pipelines (ex. Spark SQL) and machine learning algorithms (ex. MLlib);
> * Be easily deployed on cloud or on-premise: CPU & GPU, Ethernet and Infiniband.
> We propose to merge TFoS into Apache Spark as a scalable deep learning library to:
> * Make deep learning easy for Apache Spark community: Familiar pipeline API for training and inference; Enable TensorFlow training/inference on existing Spark clusters.
> * Further simplify data scientist experience: Ensure compatibility b/w Apache Spark and TFoS; Reduce steps for installation.
> * Help Apache Spark evolution on deep learning: Establish a design pattern for additional frameworks (ex. Caffe, CNTK); Structured streaming for DL training/inference.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org