You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Chris Bedford <ch...@buildlackey.com> on 2014/03/07 01:45:40 UTC
Wondering where my System.out.println()'s go, also why can't i see
results of logging at error() level to my class's slf4j logger
Hi there -
I have a problem with my println and logger output not appearing any where
i can find it when I deploy a topology to my one node
cluster. I don't have this issue when i debug the topology in local mode.
In local mode i see all the output.
As a simple example I have modified the ExclamationTopology in the storm
starter project and tried to figure out where the output
was going. My modifications are shown below (in MODIFIED
ExclamationTopology section).
>>>>>>
So, my first question is:
where should i be looking for std out output, and my logger output ?
>>>>>>
I have looked in the following places:
1) the console where i launch the topology
2) the captured standard output of nimbus and supervisor processes (that
is /tmp/*.log), given the processes are launched as below:
$STORMDIR/bin/storm nimbus > /tmp/nimbus.log 2>&1 &
$STORMDIR/bin/storm supervisor > /tmp/supervisor.log 2>&1 &
3) All logfiles generated under $STORMDIR/logs/*
>>>>>>
My second question, is how is storm internal logger output being sent to
stdout in the first place ?
>>>>>>
I am seeing lots of logger output going to my console that looks like this:
9059 [Thread-18-count] INFO backtype.storm.daemon.task - Emitting:
count default [snow, 20]
However, when I look in $STORMDIR/logback/cluster.xml , I don't see
any appender configured that would go
to stdout...
I would assume that logback would require a config line like this:
<appender name="STDOUT"
class="ch.qos.logback.core.ConsoleAppender">
But I see no such line in cluster.xml
This leads me to believe that there is some additional mechanism to
configure logging that I don't understand.
Any leads much much appreciated !
thanks
Chris
MODIFIED ExclamationTopology
public class ExclamationTopology {
public static Logger LOG = LoggerFactory.getLogger(ExclamationTopology
.class);
public static class ExclamationBolt extends BaseRichBolt {
OutputCollector _collector;
@Override
public void prepare(Map conf, TopologyContext context, OutputCollector
collector) {
_collector = collector;
}
@Override
public void execute(Tuple tuple) {
System.out.println(">>>>tuple: " + tuple);
LOG.error(">>>>tuple: logger " + tuple);
_collector.emit(tuple, new Values(tuple.getString(0) + "!!!"));
_collector.ack(tuple);
}
@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("word"));
}
}
public static void main(String[] args) throws Exception {
System.out.println(">>> STARING THE PROGRAM");
LOG.error(">>> -logger -- STARING THE PROGRAM");
TopologyBuilder builder = new TopologyBuilder();
etc. etc. the rest is the same as the 'stock version..
--
Chris Bedford
Founder & Lead Lackey
Build Lackey Labs: http://buildlackey.com
Go Grails!: http://blog.buildlackey.com
Re: Wondering where my System.out.println()'s go, also why can't i
see results of logging at error() level to my class's slf4j logger
Posted by Chris Bedford <ch...@buildlackey.com>.
Silly me.
i did not provide the "storm jar" command with a topology name, so this ran
in local mode. when i gave the starter program a topology name it ran in
remote mode, and the log files showed up in
<STORMDIR>/logs/worker-<port>.log as expected.
And the format of the logs did follow what was in
<STORMDIR>/logback/cluster.xml
The only thing I don't understand... not that it's crucial, but I am
curious.. is why when i run in LOCAL mode my log output looks like this >
102422 [Thread-18-exclaim1] ERROR storm.starter.ExclamationTopology -
>>>>tuple: logger source: word:7, stream: default, id: {}, [bertels]
but when i run remote, the statements that are captured in my log are
formatted differently >
2014-03-06 20:09:55 s.s.ExclamationTopology [ERROR] >>>>tuple: logger
source: exclaim1:4, stream: default, id: {}, [nathan!!!]
Note that the log statements emitted when running local have a [ thread
] in the pattern, whereas when running remote you see a [ date/time ] in
the pattern (as the logback/cluster.xml config would dictate ).
Just curious as to why this happens like this.
thanks !
chris
On Thu, Mar 6, 2014 at 4:45 PM, Chris Bedford <ch...@buildlackey.com> wrote:
>
>
> Hi there -
>
> I have a problem with my println and logger output not appearing any
> where i can find it when I deploy a topology to my one node
> cluster. I don't have this issue when i debug the topology in local mode.
> In local mode i see all the output.
>
> As a simple example I have modified the ExclamationTopology in the storm
> starter project and tried to figure out where the output
> was going. My modifications are shown below (in MODIFIED
> ExclamationTopology section).
>
>
>
> >>>>>>
> So, my first question is:
> where should i be looking for std out output, and my logger output ?
> >>>>>>
>
>
> I have looked in the following places:
> 1) the console where i launch the topology
>
>
> 2) the captured standard output of nimbus and supervisor processes
> (that is /tmp/*.log), given the processes are launched as below:
>
> $STORMDIR/bin/storm nimbus > /tmp/nimbus.log 2>&1 &
> $STORMDIR/bin/storm supervisor > /tmp/supervisor.log 2>&1 &
>
> 3) All logfiles generated under $STORMDIR/logs/*
>
>
>
> >>>>>>
> My second question, is how is storm internal logger output being sent to
> stdout in the first place ?
> >>>>>>
>
> I am seeing lots of logger output going to my console that looks like
> this:
> 9059 [Thread-18-count] INFO backtype.storm.daemon.task - Emitting:
> count default [snow, 20]
>
>
> However, when I look in $STORMDIR/logback/cluster.xml , I don't see
> any appender configured that would go
> to stdout...
>
> I would assume that logback would require a config line like this:
>
> <appender name="STDOUT"
> class="ch.qos.logback.core.ConsoleAppender">
>
>
> But I see no such line in cluster.xml
>
> This leads me to believe that there is some additional mechanism
> to configure logging that I don't understand.
>
> Any leads much much appreciated !
>
>
>
> thanks
> Chris
>
>
>
>
>
>
>
>
>
> MODIFIED ExclamationTopology
>
>
> public class ExclamationTopology {
>
> public static Logger LOG = LoggerFactory.getLogger(ExclamationTopology
> .class);
>
>
> public static class ExclamationBolt extends BaseRichBolt {
> OutputCollector _collector;
>
> @Override
> public void prepare(Map conf, TopologyContext context, OutputCollector
> collector) {
> _collector = collector;
> }
>
> @Override
> public void execute(Tuple tuple) {
> System.out.println(">>>>tuple: " + tuple);
> LOG.error(">>>>tuple: logger " + tuple);
> _collector.emit(tuple, new Values(tuple.getString(0) + "!!!"));
> _collector.ack(tuple);
> }
>
> @Override
> public void declareOutputFields(OutputFieldsDeclarer declarer) {
> declarer.declare(new Fields("word"));
> }
>
>
> }
>
> public static void main(String[] args) throws Exception {
> System.out.println(">>> STARING THE PROGRAM");
> LOG.error(">>> -logger -- STARING THE PROGRAM");
> TopologyBuilder builder = new TopologyBuilder();
>
>
> etc. etc. the rest is the same as the 'stock version..
>
>
>
>
>
>
> --
> Chris Bedford
>
> Founder & Lead Lackey
> Build Lackey Labs: http://buildlackey.com
> Go Grails!: http://blog.buildlackey.com
>
>
>
--
Chris Bedford
Founder & Lead Lackey
Build Lackey Labs: http://buildlackey.com
Go Grails!: http://blog.buildlackey.com