You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Nan Zhu <zh...@gmail.com> on 2015/01/09 03:33:48 UTC
missing document of several messages in actor-based receiver?
Hi, TD and other streaming developers,
When I look at the implementation of actor-based receiver (ActorReceiver.scala), I found that there are several messages which are not mentioned in the document
case props: Props =>
val worker = context.actorOf(props)
logInfo("Started receiver worker at:" + worker.path)
sender ! worker
case (props: Props, name: String) =>
val worker = context.actorOf(props, name)
logInfo("Started receiver worker at:" + worker.path)
sender ! worker
case _: PossiblyHarmful => hiccups.incrementAndGet()
case _: Statistics =>
val workers = context.children
sender ! Statistics(n.get, workers.size, hiccups.get, workers.mkString("\n”))
Is it hided with intention or incomplete document, or I missed something?
And the handler of these messages are “buggy"? e.g. when we start a new worker, we didn’t increase n (counter of children), and n and hiccups are unnecessarily set to AtomicInteger ?
Best,
--
Nan Zhu
http://codingcat.me
Re: missing document of several messages in actor-based
receiver?
Posted by Nan Zhu <zh...@gmail.com>.
Hi,
I have created the PR for these two issues
Best,
--
Nan Zhu
http://codingcat.me
On Friday, January 9, 2015 at 7:38 AM, Nan Zhu wrote:
> Thanks, TD,
>
> I just created 2 JIRAs to track these,
>
> https://issues.apache.org/jira/browse/SPARK-5174
>
> https://issues.apache.org/jira/browse/SPARK-5175
>
> Can you help to me assign these two JIRAs to me, and I’d like to submit the PRs
>
> Best,
>
> --
> Nan Zhu
> http://codingcat.me
>
>
> On Friday, January 9, 2015 at 4:25 AM, Tathagata Das wrote:
>
> > It was not really mean to be hidden. So its essentially the case of the documentation being insufficient. This code has not gotten much attention for a while, so it could have a bugs. If you find any and submit a fix for them, I am happy to take a look!
> >
> > TD
> >
> > On Thu, Jan 8, 2015 at 6:33 PM, Nan Zhu <zhunanmcgill@gmail.com (mailto:zhunanmcgill@gmail.com)> wrote:
> > > Hi, TD and other streaming developers,
> > >
> > > When I look at the implementation of actor-based receiver (ActorReceiver.scala), I found that there are several messages which are not mentioned in the document
> > >
> > > case props: Props =>
> > > val worker = context.actorOf(props)
> > > logInfo("Started receiver worker at:" + worker.path)
> > > sender ! worker
> > >
> > > case (props: Props, name: String) =>
> > > val worker = context.actorOf(props, name)
> > > logInfo("Started receiver worker at:" + worker.path)
> > > sender ! worker
> > >
> > > case _: PossiblyHarmful => hiccups.incrementAndGet()
> > >
> > > case _: Statistics =>
> > > val workers = context.children
> > > sender ! Statistics(n.get, workers.size, hiccups.get, workers.mkString("\n”))
> > >
> > > Is it hided with intention or incomplete document, or I missed something?
> > > And the handler of these messages are “buggy"? e.g. when we start a new worker, we didn’t increase n (counter of children), and n and hiccups are unnecessarily set to AtomicInteger ?
> > >
> > > Best,
> > >
> > > --
> > > Nan Zhu
> > > http://codingcat.me
> > >
> > >
> >
> >
> >
>
Re: missing document of several messages in actor-based
receiver?
Posted by Nan Zhu <zh...@gmail.com>.
Thanks, TD,
I just created 2 JIRAs to track these,
https://issues.apache.org/jira/browse/SPARK-5174
https://issues.apache.org/jira/browse/SPARK-5175
Can you help to me assign these two JIRAs to me, and I’d like to submit the PRs
Best,
--
Nan Zhu
http://codingcat.me
On Friday, January 9, 2015 at 4:25 AM, Tathagata Das wrote:
> It was not really mean to be hidden. So its essentially the case of the documentation being insufficient. This code has not gotten much attention for a while, so it could have a bugs. If you find any and submit a fix for them, I am happy to take a look!
>
> TD
>
> On Thu, Jan 8, 2015 at 6:33 PM, Nan Zhu <zhunanmcgill@gmail.com (mailto:zhunanmcgill@gmail.com)> wrote:
> > Hi, TD and other streaming developers,
> >
> > When I look at the implementation of actor-based receiver (ActorReceiver.scala), I found that there are several messages which are not mentioned in the document
> >
> > case props: Props =>
> > val worker = context.actorOf(props)
> > logInfo("Started receiver worker at:" + worker.path)
> > sender ! worker
> >
> > case (props: Props, name: String) =>
> > val worker = context.actorOf(props, name)
> > logInfo("Started receiver worker at:" + worker.path)
> > sender ! worker
> >
> > case _: PossiblyHarmful => hiccups.incrementAndGet()
> >
> > case _: Statistics =>
> > val workers = context.children
> > sender ! Statistics(n.get, workers.size, hiccups.get, workers.mkString("\n”))
> >
> > Is it hided with intention or incomplete document, or I missed something?
> > And the handler of these messages are “buggy"? e.g. when we start a new worker, we didn’t increase n (counter of children), and n and hiccups are unnecessarily set to AtomicInteger ?
> >
> > Best,
> >
> > --
> > Nan Zhu
> > http://codingcat.me
> >
> >
>
>
>
Re: missing document of several messages in actor-based receiver?
Posted by Tathagata Das <td...@databricks.com>.
It was not really mean to be hidden. So its essentially the case of the
documentation being insufficient. This code has not gotten much attention
for a while, so it could have a bugs. If you find any and submit a fix for
them, I am happy to take a look!
TD
On Thu, Jan 8, 2015 at 6:33 PM, Nan Zhu <zh...@gmail.com> wrote:
> Hi, TD and other streaming developers,
>
> When I look at the implementation of actor-based receiver
> (ActorReceiver.scala), I found that there are several messages which are
> not mentioned in the document
>
> case props: Props =>
> val worker = context.actorOf(props)
> logInfo("Started receiver worker at:" + worker.path)
> sender ! worker
>
> case (props: Props, name: String) =>
> val worker = context.actorOf(props, name)
> logInfo("Started receiver worker at:" + worker.path)
> sender ! worker
>
> case _: PossiblyHarmful => hiccups.incrementAndGet()
>
> case _: Statistics =>
> val workers = context.children
> sender ! Statistics(n.get, workers.size, hiccups.get, workers.mkString("\n*”*))
>
>
> Is it hided with intention or incomplete document, or I missed something?
>
> And the handler of these messages are “buggy"? e.g. when we start a new worker, we didn’t increase n (counter of children), and n and hiccups are unnecessarily set to AtomicInteger ?
>
> Best,
>
> --
> Nan Zhu
> http://codingcat.me
>