You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by clay teahouse <cl...@gmail.com> on 2015/01/19 12:53:38 UTC
worker logs
Hi All,
Sorry if these questions have asked before. I cannot find the answers.
1) Where do I specify the location of the worker logs? The default seems to
be /tmp.
2) How do I rotate these logs?
thanks,
Clay
Re: worker logs
Posted by clay teahouse <cl...@gmail.com>.
Even in local mode, logs are generated per worker.
On Mon, Jan 19, 2015 at 1:44 PM, Parth Brahmbhatt <
pbrahmbhatt@hortonworks.com> wrote:
> I don’t think local mode will honor your changes. Local mode does not
> spawn new worker processes. Your worker.xml changes will only take effect
> in distributed mode.
>
> Thanks
> Parth
>
> On Jan 19, 2015, at 11:34 AM, clay teahouse <cl...@gmail.com>
> wrote:
>
> I am using 0.9.3. I did set it in storm.yaml, and the console log shows
> it as set, but the worker logs still end up in /tmp.
> I am running storm in local cluster mode.
>
> thanks,
> Clay
>
> On Mon, Jan 19, 2015 at 12:39 PM, Parth Brahmbhatt <
> pbrahmbhatt@hortonworks.com> wrote:
>
>> What version of storm are you using? I just tried this locally and it
>> seem to be working on the master branch right now.
>>
>> You can try to set the “storm.log.dir” in your storm config and see if
>> that takes effect or not.
>>
>> Thanks
>> Parth
>>
>> On Jan 19, 2015, at 9:55 AM, clay teahouse <cl...@gmail.com>
>> wrote:
>>
>> Thanks Parth.
>> I did set storm.log.dir in storm.yaml, but it seems that it is being
>> ignored.
>> .
>>
>> On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt <
>> pbrahmbhatt@hortonworks.com> wrote:
>>
>>> you can specify the worker log location in worker.xml. The default
>>> is ${storm.log.dir}/${logfile.name}. You can find the file under
>>> STORM_HOME/logback/.
>>>
>>> Thanks
>>> Parth
>>>
>>>
>>>
>>> On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
>>>
>>> I'm afraid I haven't specified them. That location is where the logs
>>> were created by default.
>>>
>>> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com>
>>> wrote:
>>>
>>>> Thanks Susana.
>>>> Where do you specify the location of the logs?
>>>>
>>>> -Clay
>>>>
>>>> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Clay,
>>>>>
>>>>> I use "logrotate" to rotate the storm worker logs (in my case they are
>>>>> generated by default in $STORM_HOME/logs). You can create e.g. a file
>>>>> called "storm" in the folder /etc/logrotate.d with something like the
>>>>> following:
>>>>>
>>>>> /opt/storm/logs/worker-*.log {
>>>>> daily
>>>>> rotate 5
>>>>> compress
>>>>> missingok
>>>>> notifempty
>>>>> copytruncate
>>>>> create 640 root root
>>>>> }
>>>>>
>>>>> Regards,
>>>>> Susana
>>>>>
>>>>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <
>>>>> clayteahouse@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>> Sorry if these questions have asked before. I cannot find the answers.
>>>>>>
>>>>>> 1) Where do I specify the location of the worker logs? The default
>>>>>> seems to be /tmp.
>>>>>> 2) How do I rotate these logs?
>>>>>>
>>>>>> thanks,
>>>>>> Clay
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: worker logs
Posted by Parth Brahmbhatt <pb...@hortonworks.com>.
I don’t think local mode will honor your changes. Local mode does not spawn new worker processes. Your worker.xml changes will only take effect in distributed mode.
Thanks
Parth
On Jan 19, 2015, at 11:34 AM, clay teahouse <cl...@gmail.com> wrote:
> I am using 0.9.3. I did set it in storm.yaml, and the console log shows it as set, but the worker logs still end up in /tmp.
> I am running storm in local cluster mode.
>
> thanks,
> Clay
>
> On Mon, Jan 19, 2015 at 12:39 PM, Parth Brahmbhatt <pb...@hortonworks.com> wrote:
> What version of storm are you using? I just tried this locally and it seem to be working on the master branch right now.
>
> You can try to set the “storm.log.dir” in your storm config and see if that takes effect or not.
>
> Thanks
> Parth
>
> On Jan 19, 2015, at 9:55 AM, clay teahouse <cl...@gmail.com> wrote:
>
>> Thanks Parth.
>> I did set storm.log.dir in storm.yaml, but it seems that it is being ignored.
>> .
>>
>> On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt <pb...@hortonworks.com> wrote:
>> you can specify the worker log location in worker.xml. The default is ${storm.log.dir}/${logfile.name}. You can find the file under STORM_HOME/logback/.
>>
>> Thanks
>> Parth
>>
>>
>>
>> On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
>>
>>> I'm afraid I haven't specified them. That location is where the logs were created by default.
>>>
>>> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com> wrote:
>>> Thanks Susana.
>>> Where do you specify the location of the logs?
>>>
>>> -Clay
>>>
>>> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com> wrote:
>>> Hi Clay,
>>>
>>> I use "logrotate" to rotate the storm worker logs (in my case they are generated by default in $STORM_HOME/logs). You can create e.g. a file called "storm" in the folder /etc/logrotate.d with something like the following:
>>>
>>> /opt/storm/logs/worker-*.log {
>>> daily
>>> rotate 5
>>> compress
>>> missingok
>>> notifempty
>>> copytruncate
>>> create 640 root root
>>> }
>>>
>>> Regards,
>>> Susana
>>>
>>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com> wrote:
>>> Hi All,
>>> Sorry if these questions have asked before. I cannot find the answers.
>>>
>>> 1) Where do I specify the location of the worker logs? The default seems to be /tmp.
>>> 2) How do I rotate these logs?
>>>
>>> thanks,
>>> Clay
>>>
>>>
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: worker logs
Posted by clay teahouse <cl...@gmail.com>.
I am using 0.9.3. I did set it in storm.yaml, and the console log shows it
as set, but the worker logs still end up in /tmp.
I am running storm in local cluster mode.
thanks,
Clay
On Mon, Jan 19, 2015 at 12:39 PM, Parth Brahmbhatt <
pbrahmbhatt@hortonworks.com> wrote:
> What version of storm are you using? I just tried this locally and it seem
> to be working on the master branch right now.
>
> You can try to set the “storm.log.dir” in your storm config and see if
> that takes effect or not.
>
> Thanks
> Parth
>
> On Jan 19, 2015, at 9:55 AM, clay teahouse <cl...@gmail.com> wrote:
>
> Thanks Parth.
> I did set storm.log.dir in storm.yaml, but it seems that it is being
> ignored.
> .
>
> On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt <
> pbrahmbhatt@hortonworks.com> wrote:
>
>> you can specify the worker log location in worker.xml. The default
>> is ${storm.log.dir}/${logfile.name}. You can find the file under
>> STORM_HOME/logback/.
>>
>> Thanks
>> Parth
>>
>>
>>
>> On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
>>
>> I'm afraid I haven't specified them. That location is where the logs were
>> created by default.
>>
>> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com>
>> wrote:
>>
>>> Thanks Susana.
>>> Where do you specify the location of the logs?
>>>
>>> -Clay
>>>
>>> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com>
>>> wrote:
>>>
>>>> Hi Clay,
>>>>
>>>> I use "logrotate" to rotate the storm worker logs (in my case they are
>>>> generated by default in $STORM_HOME/logs). You can create e.g. a file
>>>> called "storm" in the folder /etc/logrotate.d with something like the
>>>> following:
>>>>
>>>> /opt/storm/logs/worker-*.log {
>>>> daily
>>>> rotate 5
>>>> compress
>>>> missingok
>>>> notifempty
>>>> copytruncate
>>>> create 640 root root
>>>> }
>>>>
>>>> Regards,
>>>> Susana
>>>>
>>>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <clayteahouse@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi All,
>>>>> Sorry if these questions have asked before. I cannot find the answers.
>>>>>
>>>>> 1) Where do I specify the location of the worker logs? The default
>>>>> seems to be /tmp.
>>>>> 2) How do I rotate these logs?
>>>>>
>>>>> thanks,
>>>>> Clay
>>>>>
>>>>
>>>>
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: worker logs
Posted by Parth Brahmbhatt <pb...@hortonworks.com>.
What version of storm are you using? I just tried this locally and it seem to be working on the master branch right now.
You can try to set the “storm.log.dir” in your storm config and see if that takes effect or not.
Thanks
Parth
On Jan 19, 2015, at 9:55 AM, clay teahouse <cl...@gmail.com> wrote:
> Thanks Parth.
> I did set storm.log.dir in storm.yaml, but it seems that it is being ignored.
> .
>
> On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt <pb...@hortonworks.com> wrote:
> you can specify the worker log location in worker.xml. The default is ${storm.log.dir}/${logfile.name}. You can find the file under STORM_HOME/logback/.
>
> Thanks
> Parth
>
>
>
> On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
>
>> I'm afraid I haven't specified them. That location is where the logs were created by default.
>>
>> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com> wrote:
>> Thanks Susana.
>> Where do you specify the location of the logs?
>>
>> -Clay
>>
>> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com> wrote:
>> Hi Clay,
>>
>> I use "logrotate" to rotate the storm worker logs (in my case they are generated by default in $STORM_HOME/logs). You can create e.g. a file called "storm" in the folder /etc/logrotate.d with something like the following:
>>
>> /opt/storm/logs/worker-*.log {
>> daily
>> rotate 5
>> compress
>> missingok
>> notifempty
>> copytruncate
>> create 640 root root
>> }
>>
>> Regards,
>> Susana
>>
>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com> wrote:
>> Hi All,
>> Sorry if these questions have asked before. I cannot find the answers.
>>
>> 1) Where do I specify the location of the worker logs? The default seems to be /tmp.
>> 2) How do I rotate these logs?
>>
>> thanks,
>> Clay
>>
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: worker logs
Posted by clay teahouse <cl...@gmail.com>.
Thanks Parth.
I did set storm.log.dir in storm.yaml, but it seems that it is being
ignored.
.
On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt <
pbrahmbhatt@hortonworks.com> wrote:
> you can specify the worker log location in worker.xml. The default
> is ${storm.log.dir}/${logfile.name}. You can find the file under
> STORM_HOME/logback/.
>
> Thanks
> Parth
>
>
>
> On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
>
> I'm afraid I haven't specified them. That location is where the logs were
> created by default.
>
> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com>
> wrote:
>
>> Thanks Susana.
>> Where do you specify the location of the logs?
>>
>> -Clay
>>
>> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com>
>> wrote:
>>
>>> Hi Clay,
>>>
>>> I use "logrotate" to rotate the storm worker logs (in my case they are
>>> generated by default in $STORM_HOME/logs). You can create e.g. a file
>>> called "storm" in the folder /etc/logrotate.d with something like the
>>> following:
>>>
>>> /opt/storm/logs/worker-*.log {
>>> daily
>>> rotate 5
>>> compress
>>> missingok
>>> notifempty
>>> copytruncate
>>> create 640 root root
>>> }
>>>
>>> Regards,
>>> Susana
>>>
>>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>> Sorry if these questions have asked before. I cannot find the answers.
>>>>
>>>> 1) Where do I specify the location of the worker logs? The default
>>>> seems to be /tmp.
>>>> 2) How do I rotate these logs?
>>>>
>>>> thanks,
>>>> Clay
>>>>
>>>
>>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: worker logs
Posted by Parth Brahmbhatt <pb...@hortonworks.com>.
you can specify the worker log location in worker.xml. The default is ${storm.log.dir}/${logfile.name}. You can find the file under STORM_HOME/logback/.
Thanks
Parth
On Jan 19, 2015, at 7:34 AM, Susana González <su...@gmail.com> wrote:
> I'm afraid I haven't specified them. That location is where the logs were created by default.
>
> On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com> wrote:
> Thanks Susana.
> Where do you specify the location of the logs?
>
> -Clay
>
> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com> wrote:
> Hi Clay,
>
> I use "logrotate" to rotate the storm worker logs (in my case they are generated by default in $STORM_HOME/logs). You can create e.g. a file called "storm" in the folder /etc/logrotate.d with something like the following:
>
> /opt/storm/logs/worker-*.log {
> daily
> rotate 5
> compress
> missingok
> notifempty
> copytruncate
> create 640 root root
> }
>
> Regards,
> Susana
>
> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com> wrote:
> Hi All,
> Sorry if these questions have asked before. I cannot find the answers.
>
> 1) Where do I specify the location of the worker logs? The default seems to be /tmp.
> 2) How do I rotate these logs?
>
> thanks,
> Clay
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: worker logs
Posted by Susana González <su...@gmail.com>.
I'm afraid I haven't specified them. That location is where the logs were
created by default.
On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse <cl...@gmail.com>
wrote:
> Thanks Susana.
> Where do you specify the location of the logs?
>
> -Clay
>
> On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com>
> wrote:
>
>> Hi Clay,
>>
>> I use "logrotate" to rotate the storm worker logs (in my case they are
>> generated by default in $STORM_HOME/logs). You can create e.g. a file
>> called "storm" in the folder /etc/logrotate.d with something like the
>> following:
>>
>> /opt/storm/logs/worker-*.log {
>> daily
>> rotate 5
>> compress
>> missingok
>> notifempty
>> copytruncate
>> create 640 root root
>> }
>>
>> Regards,
>> Susana
>>
>> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>> Sorry if these questions have asked before. I cannot find the answers.
>>>
>>> 1) Where do I specify the location of the worker logs? The default seems
>>> to be /tmp.
>>> 2) How do I rotate these logs?
>>>
>>> thanks,
>>> Clay
>>>
>>
>>
>
Re: worker logs
Posted by clay teahouse <cl...@gmail.com>.
Thanks Susana.
Where do you specify the location of the logs?
-Clay
On Mon, Jan 19, 2015 at 9:09 AM, Susana González <su...@gmail.com> wrote:
> Hi Clay,
>
> I use "logrotate" to rotate the storm worker logs (in my case they are
> generated by default in $STORM_HOME/logs). You can create e.g. a file
> called "storm" in the folder /etc/logrotate.d with something like the
> following:
>
> /opt/storm/logs/worker-*.log {
> daily
> rotate 5
> compress
> missingok
> notifempty
> copytruncate
> create 640 root root
> }
>
> Regards,
> Susana
>
> On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com>
> wrote:
>
>> Hi All,
>> Sorry if these questions have asked before. I cannot find the answers.
>>
>> 1) Where do I specify the location of the worker logs? The default seems
>> to be /tmp.
>> 2) How do I rotate these logs?
>>
>> thanks,
>> Clay
>>
>
>
Topology dead even with synchronised output collector
Posted by Andrew Wu <aw...@innometrics.com>.
Hi Guys
I have fight over this for several days now.
34493 [Thread-17-disruptor-executor[3 8]-send-queue] ERROR backtype.storm.util - Async loop died!
java.lang.RuntimeException: java.lang.NullPointerException
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.disruptor$consume_loop_STAR_$fn__1460.invoke(disruptor.clj:94) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:724) [na:1.7.0_40]
Caused by: java.lang.NullPointerException: null
at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$mk_transfer_fn$fn__3549.invoke(worker.clj:129) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3283.invoke(executor.clj:258) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.3.jar:0.9.3]
My research shows this should have been caused by OutputCollector being accessed by multiple threads.
I tried both using synchronised block and volatile keyword, this problem persists.
As described in https://storm.apache.org/documentation/Concepts.html OutputCollector should thread safe.
Question is: Is this OutputCollector thread safe? if so, what else may cause this problem?
Many thanks!
Andrew Ge Wu
Backend Developer & Integration Tech Lead
Mobile: +46 76 88 99 970
Website: www.innometrics.com<http://www.innometrics.com/>
[cid:A29501B8-EA20-463B-A4A4-203BB3DA4788]
Re: worker logs
Posted by Susana González <su...@gmail.com>.
Hi Clay,
I use "logrotate" to rotate the storm worker logs (in my case they are
generated by default in $STORM_HOME/logs). You can create e.g. a file
called "storm" in the folder /etc/logrotate.d with something like the
following:
/opt/storm/logs/worker-*.log {
daily
rotate 5
compress
missingok
notifempty
copytruncate
create 640 root root
}
Regards,
Susana
On Mon, Jan 19, 2015 at 12:53 PM, clay teahouse <cl...@gmail.com>
wrote:
> Hi All,
> Sorry if these questions have asked before. I cannot find the answers.
>
> 1) Where do I specify the location of the worker logs? The default seems
> to be /tmp.
> 2) How do I rotate these logs?
>
> thanks,
> Clay
>