You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Abhishek Agarwal <ab...@gmail.com> on 2015/09/08 12:20:00 UTC
Overriding Log4j configuration per topology
Right now, AFAIK there is a fixed log4j configuration which is shared
across all topologies in a cluster. This is currently encoded in
worker.xml. I was wondering if we can have a way to override this
configuration per topology
Pros
> I could have a custom retention policy for each of my topology. Some
applications may need a size based and file count based retention. Others
may need time based retention
> I could filter out the logs of a particular package into separate log
file. It may not be possible to do so in the cluster wide log configuration
Many more customizations are possible of course.
Cons
> Log management and retention could become a problem for the cluster admin.
> There are security implications and permissions on log directories need
to be correct.
Thoughts/comments?
--
Regards,
Abhishek Agarwal
Re: Overriding Log4j configuration per topology
Posted by Abhishek Agarwal <ab...@gmail.com>.
My requirement is different from turning on debugging in runtime. I want to
pass the worker.xml in my topology jar. Worker of my topology should pick
up the file in user jar instead of $STORM_HOME/log4j/worker.xml
On Wed, Sep 9, 2015 at 12:06 AM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> What exactly do you want to change?
> We have a JIRA https://issues.apache.org/jira/browse/STORM-412 that we
> have working code internally for, and should be pushing back to open source
> soon. Is that enough of what you want to do, or do you want to configure
> where the logs go?
> - Bobby
>
>
> On Tuesday, September 8, 2015 5:20 AM, Abhishek Agarwal <
> abhishcool@gmail.com> wrote:
>
>
> Right now, AFAIK there is a fixed log4j configuration which is shared
> across all topologies in a cluster. This is currently encoded in
> worker.xml. I was wondering if we can have a way to override this
> configuration per topology
>
> Pros
> > I could have a custom retention policy for each of my topology. Some
> applications may need a size based and file count based retention. Others
> may need time based retention
> > I could filter out the logs of a particular package into separate log
> file. It may not be possible to do so in the cluster wide log configuration
>
> Many more customizations are possible of course.
>
> Cons
> > Log management and retention could become a problem for the cluster
> admin.
> > There are security implications and permissions on log directories need
> to be correct.
>
> Thoughts/comments?
> --
> Regards,
> Abhishek Agarwal
>
>
>
>
--
Regards,
Abhishek Agarwal
Re: Overriding Log4j configuration per topology
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
What exactly do you want to change?
We have a JIRA https://issues.apache.org/jira/browse/STORM-412 that we have working code internally for, and should be pushing back to open source soon. Is that enough of what you want to do, or do you want to configure where the logs go?
- Bobby
On Tuesday, September 8, 2015 5:20 AM, Abhishek Agarwal <ab...@gmail.com> wrote:
Right now, AFAIK there is a fixed log4j configuration which is shared
across all topologies in a cluster. This is currently encoded in
worker.xml. I was wondering if we can have a way to override this
configuration per topology
Pros
> I could have a custom retention policy for each of my topology. Some
applications may need a size based and file count based retention. Others
may need time based retention
> I could filter out the logs of a particular package into separate log
file. It may not be possible to do so in the cluster wide log configuration
Many more customizations are possible of course.
Cons
> Log management and retention could become a problem for the cluster admin.
> There are security implications and permissions on log directories need
to be correct.
Thoughts/comments?
--
Regards,
Abhishek Agarwal