You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by OrielResearch Eila Arich-Landkof <ei...@orielresearch.org> on 2018/04/10 23:27:37 UTC
Passing parameter to DoFn in Python
Hi all,
Is it possible to pass a string parameter with DoFn function and what would
be the syntax?
The call should look something like that:
beam.ParDo(SampleFn(samplePath))
how would the class definition be updated?
class SampleFn(beam.DoFn):
def process(self,element):
Thanks,
--
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/
Re: Passing parameter to DoFn in Python
Posted by OrielResearch Eila Arich-Landkof <ei...@orielresearch.org>.
great! thanks
On Tue, Apr 10, 2018 at 7:31 PM, Robert Bradshaw <ro...@google.com>
wrote:
> Yes, DoFns are normal Python classes. To do this you would write
>
> class SampleFn(beam.DoFn):
> def __init__(self, samplePath):
> self.samplePath = samplePath
>
> def process(self, element):
> # use self.samplePath here, will get to remote workers via pickling
>
> On Tue, Apr 10, 2018 at 4:27 PM OrielResearch Eila Arich-Landkof <
> eila@orielresearch.org> wrote:
>
>> Hi all,
>>
>> Is it possible to pass a string parameter with DoFn function and what
>> would be the syntax?
>>
>> The call should look something like that:
>> beam.ParDo(SampleFn(samplePath))
>>
>> how would the class definition be updated?
>> class SampleFn(beam.DoFn):
>> def process(self,element):
>>
>>
>> Thanks,
>>
>> --
>> Eila
>> www.orielresearch.org
>> https://www.meetup.com/Deep-Learning-In-Production/
>>
>
--
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/
Re: Passing parameter to DoFn in Python
Posted by Robert Bradshaw <ro...@google.com>.
Yes, DoFns are normal Python classes. To do this you would write
class SampleFn(beam.DoFn):
def __init__(self, samplePath):
self.samplePath = samplePath
def process(self, element):
# use self.samplePath here, will get to remote workers via pickling
On Tue, Apr 10, 2018 at 4:27 PM OrielResearch Eila Arich-Landkof <
eila@orielresearch.org> wrote:
> Hi all,
>
> Is it possible to pass a string parameter with DoFn function and what
> would be the syntax?
>
> The call should look something like that:
> beam.ParDo(SampleFn(samplePath))
>
> how would the class definition be updated?
> class SampleFn(beam.DoFn):
> def process(self,element):
>
>
> Thanks,
>
> --
> Eila
> www.orielresearch.org
> https://www.meetup.com/Deep-Learning-In-Production/
>