You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Paolo Patierno <pp...@live.com> on 2016/06/24 13:07:19 UTC

Logging trait in Spark 2.0

Hi,

developing a Spark Streaming custom receiver I noticed that the Logging trait isn't accessible anymore in Spark 2.0.

trait Logging in package internal cannot be accessed in package org.apache.spark.internal

For developing a custom receiver what is the preferred way for logging ? Just using log4j dependency as any other Java/Scala library/application ?

Thanks,
Paolo

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

Re: Logging trait in Spark 2.0

Posted by Stephen Boesch <ja...@gmail.com>.
I also did not understand why the Logging class was made private in Spark
2.0.  In a couple of projects including CaffeOnSpark the Logging class was
simply copied to the new project to allow for backwards compatibility.

2016-06-28 18:10 GMT-07:00 Michael Armbrust <mi...@databricks.com>:

> I'd suggest using the slf4j APIs directly.  They provide a nice stable API
> that works with a variety of logging backends.  This is what Spark does
> internally.
>
> On Sun, Jun 26, 2016 at 4:02 AM, Paolo Patierno <pp...@live.com>
> wrote:
>
>> Yes ... the same here ... I'd like to know the best way for adding
>> logging in a custom receiver for Spark Streaming 2.0
>>
>> *Paolo Patierno*
>>
>> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
>> Embedded & IoT*
>> *Microsoft Azure Advisor*
>>
>> Twitter : @ppatierno <http://twitter.com/ppatierno>
>> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
>> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>>
>>
>> ------------------------------
>> From: jonathakamzn@gmail.com
>> Date: Fri, 24 Jun 2016 20:56:40 +0000
>> Subject: Re: Logging trait in Spark 2.0
>> To: yuzhihong@gmail.com; ppatierno@live.com
>> CC: user@spark.apache.org
>>
>>
>> Ted, how is that thread related to Paolo's question?
>>
>> On Fri, Jun 24, 2016 at 1:50 PM Ted Yu <yu...@gmail.com> wrote:
>>
>> See this related thread:
>>
>>
>> http://search-hadoop.com/m/q3RTtEor1vYWbsW&subj=RE+Configuring+Log4J+Spark+1+5+on+EMR+4+1+
>>
>> On Fri, Jun 24, 2016 at 6:07 AM, Paolo Patierno <pp...@live.com>
>> wrote:
>>
>> Hi,
>>
>> developing a Spark Streaming custom receiver I noticed that the Logging
>> trait isn't accessible anymore in Spark 2.0.
>>
>> trait Logging in package internal cannot be accessed in package
>> org.apache.spark.internal
>>
>> For developing a custom receiver what is the preferred way for logging ?
>> Just using log4j dependency as any other Java/Scala library/application ?
>>
>> Thanks,
>> Paolo
>>
>> *Paolo Patierno*
>>
>> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
>> Embedded & IoT*
>> *Microsoft Azure Advisor*
>>
>> Twitter : @ppatierno <http://twitter.com/ppatierno>
>> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
>> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>>
>>
>>
>

Re: Logging trait in Spark 2.0

Posted by Michael Armbrust <mi...@databricks.com>.
I'd suggest using the slf4j APIs directly.  They provide a nice stable API
that works with a variety of logging backends.  This is what Spark does
internally.

On Sun, Jun 26, 2016 at 4:02 AM, Paolo Patierno <pp...@live.com> wrote:

> Yes ... the same here ... I'd like to know the best way for adding logging
> in a custom receiver for Spark Streaming 2.0
>
> *Paolo Patierno*
>
> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
> Embedded & IoT*
> *Microsoft Azure Advisor*
>
> Twitter : @ppatierno <http://twitter.com/ppatierno>
> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>
>
> ------------------------------
> From: jonathakamzn@gmail.com
> Date: Fri, 24 Jun 2016 20:56:40 +0000
> Subject: Re: Logging trait in Spark 2.0
> To: yuzhihong@gmail.com; ppatierno@live.com
> CC: user@spark.apache.org
>
>
> Ted, how is that thread related to Paolo's question?
>
> On Fri, Jun 24, 2016 at 1:50 PM Ted Yu <yu...@gmail.com> wrote:
>
> See this related thread:
>
>
> http://search-hadoop.com/m/q3RTtEor1vYWbsW&subj=RE+Configuring+Log4J+Spark+1+5+on+EMR+4+1+
>
> On Fri, Jun 24, 2016 at 6:07 AM, Paolo Patierno <pp...@live.com>
> wrote:
>
> Hi,
>
> developing a Spark Streaming custom receiver I noticed that the Logging
> trait isn't accessible anymore in Spark 2.0.
>
> trait Logging in package internal cannot be accessed in package
> org.apache.spark.internal
>
> For developing a custom receiver what is the preferred way for logging ?
> Just using log4j dependency as any other Java/Scala library/application ?
>
> Thanks,
> Paolo
>
> *Paolo Patierno*
>
> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
> Embedded & IoT*
> *Microsoft Azure Advisor*
>
> Twitter : @ppatierno <http://twitter.com/ppatierno>
> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>
>
>

RE: Logging trait in Spark 2.0

Posted by Paolo Patierno <pp...@live.com>.
Yes ... the same here ... I'd like to know the best way for adding logging in a custom receiver for Spark Streaming 2.0

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

From: jonathakamzn@gmail.com
Date: Fri, 24 Jun 2016 20:56:40 +0000
Subject: Re: Logging trait in Spark 2.0
To: yuzhihong@gmail.com; ppatierno@live.com
CC: user@spark.apache.org

Ted, how is that thread related to Paolo's question?
On Fri, Jun 24, 2016 at 1:50 PM Ted Yu <yu...@gmail.com> wrote:
See this related thread:
http://search-hadoop.com/m/q3RTtEor1vYWbsW&subj=RE+Configuring+Log4J+Spark+1+5+on+EMR+4+1+

On Fri, Jun 24, 2016 at 6:07 AM, Paolo Patierno <pp...@live.com> wrote:



Hi,

developing a Spark Streaming custom receiver I noticed that the Logging trait isn't accessible anymore in Spark 2.0.

trait Logging in package internal cannot be accessed in package org.apache.spark.internal

For developing a custom receiver what is the preferred way for logging ? Just using log4j dependency as any other Java/Scala library/application ?

Thanks,
Paolo

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

Re: Logging trait in Spark 2.0

Posted by Jonathan Kelly <jo...@gmail.com>.
Ted, how is that thread related to Paolo's question?

On Fri, Jun 24, 2016 at 1:50 PM Ted Yu <yu...@gmail.com> wrote:

> See this related thread:
>
>
> http://search-hadoop.com/m/q3RTtEor1vYWbsW&subj=RE+Configuring+Log4J+Spark+1+5+on+EMR+4+1+
>
> On Fri, Jun 24, 2016 at 6:07 AM, Paolo Patierno <pp...@live.com>
> wrote:
>
>> Hi,
>>
>> developing a Spark Streaming custom receiver I noticed that the Logging
>> trait isn't accessible anymore in Spark 2.0.
>>
>> trait Logging in package internal cannot be accessed in package
>> org.apache.spark.internal
>>
>> For developing a custom receiver what is the preferred way for logging ?
>> Just using log4j dependency as any other Java/Scala library/application ?
>>
>> Thanks,
>> Paolo
>>
>> *Paolo Patierno*
>>
>> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
>> Embedded & IoT*
>> *Microsoft Azure Advisor*
>>
>> Twitter : @ppatierno <http://twitter.com/ppatierno>
>> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
>> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>>
>
>

Re: Logging trait in Spark 2.0

Posted by Ted Yu <yu...@gmail.com>.
See this related thread:

http://search-hadoop.com/m/q3RTtEor1vYWbsW&subj=RE+Configuring+Log4J+Spark+1+5+on+EMR+4+1+

On Fri, Jun 24, 2016 at 6:07 AM, Paolo Patierno <pp...@live.com> wrote:

> Hi,
>
> developing a Spark Streaming custom receiver I noticed that the Logging
> trait isn't accessible anymore in Spark 2.0.
>
> trait Logging in package internal cannot be accessed in package
> org.apache.spark.internal
>
> For developing a custom receiver what is the preferred way for logging ?
> Just using log4j dependency as any other Java/Scala library/application ?
>
> Thanks,
> Paolo
>
> *Paolo Patierno*
>
> *Senior Software Engineer (IoT) @ Red Hat**Microsoft MVP on **Windows
> Embedded & IoT*
> *Microsoft Azure Advisor*
>
> Twitter : @ppatierno <http://twitter.com/ppatierno>
> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience <http://paolopatierno.wordpress.com/>
>