You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Antony Mayi <an...@yahoo.com> on 2017/04/05 14:22:37 UTC

how to getRunner() after deprecation of Pipeline.getOptions()

reading through BEAM-818 and https://docs.google.com/document/d/1Wr05cYdqnCfrLLqSk--XmGMGgDwwNwWZaFbxLKvPqEQ/edit#
if I am currently doing something like this:public class MyTransform extends PTransform<String, String> {
    @Override public String expand(PCollection<String> input) {
        Class<? extends PipelineRunner<?>> runner = input.getPipeline().getOptions().getRunner();
is the expected way to obtain the runner class after the Pipeline.getOptions() is deprecated to pass it via MyTransform constructor and keeping a reference to it?

thanks Antony.

Re: how to getRunner() after deprecation of Pipeline.getOptions()

Posted by Antony Mayi <an...@yahoo.com>.
I am just checking what is the runner class - as per the options.setRunner() assigned earlier, not accessing the runner itself...
having alternative implementation of the transformer in case the runner is a DirectRunner (if options.getRunner() == DirectRunner.class)
a. 

    On Wednesday, 5 April 2017, 16:15, Kenneth Knowles <kl...@google.com> wrote:
 

 Accessing the runner in your transform's expand() method can't work, in general: Your transform is / will be expanded to build a runner-independent graph before a runner is selected.
I'm curious - what is it that you are using it for?

On Wed, Apr 5, 2017 at 7:22 AM, Antony Mayi <an...@yahoo.com> wrote:

reading through BEAM-818 and https://docs.google.com/ document/d/1Wr05cYdqnCfrLLqSk- -XmGMGgDwwNwWZaFbxLKvPqEQ/ edit#
if I am currently doing something like this:public class MyTransform extends PTransform<String, String> {
    @Override public String expand(PCollection<String> input) {
        Class<? extends PipelineRunner<?>> runner = input.getPipeline(). getOptions().getRunner();
is the expected way to obtain the runner class after the Pipeline.getOptions() is deprecated to pass it via MyTransform constructor and keeping a reference to it?

thanks Antony.

Re: how to getRunner() after deprecation of Pipeline.getOptions()

Posted by Kenneth Knowles <kl...@google.com>.
Accessing the runner in your transform's expand() method can't work, in
general: Your transform is / will be expanded to build a runner-independent
graph before a runner is selected.

I'm curious - what is it that you are using it for?

On Wed, Apr 5, 2017 at 7:22 AM, Antony Mayi <an...@yahoo.com> wrote:

> reading through BEAM-818 and https://docs.google.com/
> document/d/1Wr05cYdqnCfrLLqSk--XmGMGgDwwNwWZaFbxLKvPqEQ/edit#
>
> if I am currently doing something like this:
>
> public class MyTransform extends PTransform<String, String> {
>     @Override public String expand(PCollection<String> input) {
>         Class<? extends PipelineRunner<?>> runner = input.getPipeline().getOptions().getRunner();
>
>
> is the expected way to obtain the runner class after the
> Pipeline.getOptions() is deprecated to pass it via MyTransform constructor
> and keeping a reference to it?
>
> thanks Antony.
>