You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Rakesh Kumar <ra...@lyft.com> on 2018/08/25 19:58:09 UTC

Tensor Flow support on Flink Runner

Hi all,

I would like to know whether Tensor Flow is supported on Flink Runner &
DirectRunner? If it is not currently supported then what is the plan to
provide the support.

Thank you
Rakesh
-- 
Rakesh Kumar
Software Engineer
510-761-1364 |

<https://www.lyft.com/>

Re: Tensor Flow support on Flink Runner

Posted by Rakesh Kumar <ra...@lyft.com>.

Thank you Ankur for the response and providing the usefult links.


On Sat, Aug 25, 2018 at 3:55 PM Ankur Goenka <go...@google.com> wrote:

> Hi Rakesh,
>
> You can use DirectRunner to run Tensorflow on Beam. Please refer
> https://www.tensorflow.org/tfx/transform/get_started to learn more about
> it.
> We are working on making Tensorflow available on Flink Runner. You can
> build Beam from Master and run the Tensorflow model on it.
> Here are general instructions to run Portable FlinkRunner
> https://beam.apache.org/contribute/portability/#status
> Please add the following parameters "--experiments= worker_threads=100"
> and "--experiments=beam_fnapi" to your pipeline arguments.
> For reference, here is a sample of TensorFlow Job on FlinkRunner
> https://github.com/angoenka/model-analysis/blob/axelmagn-hacks/examples/chicago_taxi/preprocess_flink.sh
> Note, There might be some rough edges as the PortableFlinkRunner is still
> in development.
>
> Thanks,
> Ankur
>
>
> On Sat, Aug 25, 2018 at 12:58 PM Rakesh Kumar <ra...@lyft.com>
> wrote:
>
>> Hi all,
>>
>> I would like to know whether Tensor Flow is supported on Flink Runner &
>> DirectRunner? If it is not currently supported then what is the plan to
>> provide the support.
>>
>> Thank you
>> Rakesh
>> --
>> Rakesh Kumar
>> Software Engineer
>> 510-761-1364 <(510)%20761-1364> |
>>
>> <https://www.lyft.com/>
>>
> --
Rakesh Kumar
Software Engineer
510-761-1364 |

<https://www.lyft.com/>

Re: Tensor Flow support on Flink Runner

Posted by Ankur Goenka <go...@google.com>.

Hi Rakesh,

You can use DirectRunner to run Tensorflow on Beam. Please refer
https://www.tensorflow.org/tfx/transform/get_started to learn more about it.
We are working on making Tensorflow available on Flink Runner. You can
build Beam from Master and run the Tensorflow model on it.
Here are general instructions to run Portable FlinkRunner
https://beam.apache.org/contribute/portability/#status
Please add the following parameters "--experiments= worker_threads=100" and
"--experiments=beam_fnapi" to your pipeline arguments.
For reference, here is a sample of TensorFlow Job on FlinkRunner
https://github.com/angoenka/model-analysis/blob/axelmagn-hacks/examples/chicago_taxi/preprocess_flink.sh
Note, There might be some rough edges as the PortableFlinkRunner is still
in development.

Thanks,
Ankur

On Sat, Aug 25, 2018 at 12:58 PM Rakesh Kumar <ra...@lyft.com> wrote:

> Hi all,
>
> I would like to know whether Tensor Flow is supported on Flink Runner &
> DirectRunner? If it is not currently supported then what is the plan to
> provide the support.
>
> Thank you
> Rakesh
> --
> Rakesh Kumar
> Software Engineer
> 510-761-1364 |
>
> <https://www.lyft.com/>
>