You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by eduardocalfaia <e....@unibs.it> on 2014/03/05 16:23:16 UTC
Re: Explain About Logs NetworkWordcount.scala
Hi TD,
I have seen in the web UI the stage number that result has been zero and in
the field GC Times there is nothing.
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n2306/CaptureStage.png>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2306.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Explain About Logs NetworkWordcount.scala
Posted by Eduardo Costa Alfaia <e....@unibs.it>.
Yes TD,
I can use tcpdump to see if the data are being accepted by the receiver
and if else them are arriving into the IP packet.
Thanks
Em 3/8/14, 4:19, Tathagata Das escreveu:
> I am not sure how to debug this without any more information about the
> source. Can you monitor on the receiver side that data is being
> accepted by the receiver but not reported?
>
> TD
>
>
> On Wed, Mar 5, 2014 at 7:23 AM, eduardocalfaia <e.costaalfaia@unibs.it
> <ma...@unibs.it>> wrote:
>
> Hi TD,
> I have seen in the web UI the stage number that result has been
> zero and in
> the field GC Times there is nothing.
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n2306/CaptureStage.png>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2306.html
> Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
>
>
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
Re: Explain About Logs NetworkWordcount.scala
Posted by eduardocalfaia <e....@unibs.it>.
Hi TD,
Today I have seen these differences between the logs:
Result different from zero:
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Removed
input-0-1394445287000 on computer1.ant-net:51441 in memory (size: 10.1
MB, free: 1447.3 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1394445315400 in memory on computer1.ant-net:51441 (size: 4.8
MB, free: 1442.5 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1394445315600 in memory on computer1.ant-net:51441 (size: 3.8
MB, free: 1438.8 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Removed
input-0-1394445287200 on computer1.ant-net:51441 in memory (size: 4.7
MB, free: 1448.2 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1394445315800 in memory on computer1.ant-net:51441 (size: 9.8
MB, free: 1438.4 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Removed
input-0-1394445287400 on computer1.ant-net:51441 in memory (size: 4.7
MB, free: 1447.7 MB)
14/03/10 10:55:27 INFO BlockManagerMasterActor$BlockManagerInfo: Added
input-0-1394445316000 in memory on computer1.ant-net:51441 (size: 2.5
MB, free: 1445.2 MB)
When the result is equal zero the information above not exist.
Do you have any idea about this?
BR
Em 3/8/14, 4:21, Tathagata Das [via Apache Spark User List] escreveu:
> I am not sure how to debug this without any more information about the
> source. Can you monitor on the receiver side that data is being
> accepted by the receiver but not reported?
>
> TD
>
>
> On Wed, Mar 5, 2014 at 7:23 AM, eduardocalfaia <[hidden email]
> </user/SendEmail.jtp?type=node&node=2421&i=0>> wrote:
>
> Hi TD,
> I have seen in the web UI the stage number that result has been
> zero and in
> the field GC Times there is nothing.
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n2306/CaptureStage.png>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2306.html
> Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
>
>
>
>
> ------------------------------------------------------------------------
> If you reply to this email, your message will be added to the
> discussion below:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2421.html
>
> To unsubscribe from Explain About Logs NetworkWordcount.scala, click
> here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1835&code=ZS5jb3N0YWFsZmFpYUB1bmlicy5pdHwxODM1fC0yMTE1OTc4OTYz>.
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2458.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Explain About Logs NetworkWordcount.scala
Posted by eduardocalfaia <e....@unibs.it>.
Hi TD,
I have attached a source code from the application that I use to send
the words to workers.
BR
Em 3/8/14, 4:21, Tathagata Das [via Apache Spark User List] escreveu:
> I am not sure how to debug this without any more information about the
> source. Can you monitor on the receiver side that data is being
> accepted by the receiver but not reported?
>
> TD
>
>
> On Wed, Mar 5, 2014 at 7:23 AM, eduardocalfaia <[hidden email]
> </user/SendEmail.jtp?type=node&node=2421&i=0>> wrote:
>
> Hi TD,
> I have seen in the web UI the stage number that result has been
> zero and in
> the field GC Times there is nothing.
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n2306/CaptureStage.png>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2306.html
> Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
>
>
>
>
> ------------------------------------------------------------------------
> If you reply to this email, your message will be added to the
> discussion below:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2421.html
>
> To unsubscribe from Explain About Logs NetworkWordcount.scala, click
> here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1835&code=ZS5jb3N0YWFsZmFpYUB1bmlicy5pdHwxODM1fC0yMTE1OTc4OTYz>.
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
cachesender.c (10K) <http://apache-spark-user-list.1001560.n3.nabble.com/attachment/2452/0/cachesender.c>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2452.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Explain About Logs NetworkWordcount.scala
Posted by Tathagata Das <ta...@gmail.com>.
I am not sure how to debug this without any more information about the
source. Can you monitor on the receiver side that data is being accepted by
the receiver but not reported?
TD
On Wed, Mar 5, 2014 at 7:23 AM, eduardocalfaia <e....@unibs.it>wrote:
> Hi TD,
> I have seen in the web UI the stage number that result has been zero and in
> the field GC Times there is nothing.
> <
> http://apache-spark-user-list.1001560.n3.nabble.com/file/n2306/CaptureStage.png
> >
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Explain-About-Logs-NetworkWordcount-scala-tp1835p2306.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>