You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by madhu phatak <ph...@gmail.com> on 2015/03/16 07:14:49 UTC
why generateJob is a private API?
Hi,
I am trying to create a simple subclass of DStream. If I understand
correctly, I should override *compute *lazy operations and *generateJob*
for actions. But when I try to override, generateJob it gives error saying
method is private to the streaming package. Is my approach is correct or am
I missing something?
Regards,
Madhukara Phatak
http://datamantra.io/
Re: why generateJob is a private API?
Posted by madhu phatak <ph...@gmail.com>.
Hi,
Thank you for the response.
Regards,
Madhukara Phatak
http://datamantra.io/
On Tue, Mar 17, 2015 at 5:50 AM, Tathagata Das <td...@databricks.com> wrote:
> It was not really meant to be pubic and overridden. Because anything you
> want to do to generate jobs from RDDs can be done using DStream.foreachRDD
>
> On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak <ph...@gmail.com>
> wrote:
>
>> Hi,
>> I am trying to create a simple subclass of DStream. If I understand
>> correctly, I should override *compute *lazy operations and *generateJob*
>> for actions. But when I try to override, generateJob it gives error saying
>> method is private to the streaming package. Is my approach is correct or am
>> I missing something?
>>
>>
>> Regards,
>> Madhukara Phatak
>> http://datamantra.io/
>>
>
>
Re: why generateJob is a private API?
Posted by Tathagata Das <td...@databricks.com>.
It was not really meant to be pubic and overridden. Because anything you
want to do to generate jobs from RDDs can be done using DStream.foreachRDD
On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak <ph...@gmail.com> wrote:
> Hi,
> I am trying to create a simple subclass of DStream. If I understand
> correctly, I should override *compute *lazy operations and *generateJob*
> for actions. But when I try to override, generateJob it gives error saying
> method is private to the streaming package. Is my approach is correct or am
> I missing something?
>
>
> Regards,
> Madhukara Phatak
> http://datamantra.io/
>