You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by ignite_user2016 <ri...@gmail.com> on 2017/03/10 03:08:05 UTC
Aggregation of log files
Currently, we have a monitor in place to monitor ignite servers, however we
are seeing log files for every session that is connected to ignite.As a
result, we see almost 50-80K log files every day.
Is there a way where I can aggregate log files with some settings ? any
example would be helpful here.
Looking forward for your quick help here.
Thanks,
Rishi
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
I believe IgniteVisor would act as a client to ignite server, correct ?
So wondering how to control log files here when ignite visor is used for
monitoring ?
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11131.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by vkulichenko <va...@gmail.com>.
Is this a client? If so, it's a bad practice to create a new one for each
operations. Start it once within your application and reuse. It's thread
safe and can be used concurrently as well.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11129.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
Hi Val,
We generally see the message -
Ignite node started OK (id=cc119e30) on the console
so now log file will be generated as follows -
ignite-cc119e30.0
now if I am monitoring ignite visor command every time it will connect to
ignite via new session with new id as a result for every monitor we set
would generate log file with that session id.
hope I clarify here.
Thanks,
Risih
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11128.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by vkulichenko <va...@gmail.com>.
Can you clarify what you mean by "file per session"? What is session?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11123.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
So The default logger is JUL how can I change it to log4j-2 logging ?
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11127.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
Here is the log4j config, I changed it to warn level still sees INFO
<Loggers>
<Logger name="org.apache.ignite" level="WARN"/>
<Logger name="org.springframework" level="WARN"/>
<Logger name="org.eclipse.jetty" level="WARN"/>
<Logger name="org.eclipse.jetty.util.log" level="ERROR"/>
<Logger name="org.eclipse.jetty.util.component" level="ERROR"/>
<Logger name="com.amazonaws" level="WARN"/>
<Root level="INFO">
<AppenderRef ref="CONSOLE_ERR" level="ERROR"/>
<AppenderRef ref="FILE" level="WARN"/>
</Root>
</Loggers>
[09:20:42,208][INFO][main][IgniteKernal] Config URL:
file:/C:/projects/Ignite/apache-ignite-fabric-1.7.0-bin/config/default-config.xml
[09:20:42,208][INFO][main][IgniteKernal] Daemon mode: off
[09:20:42,208][INFO][main][IgniteKernal] OS: Windows 7 6.1 x86
[09:20:42,208][INFO][main][IgniteKernal] OS user: host
[09:20:42,208][INFO][main][IgniteKernal] Language runtime: Java Platform API
Specification ver. 1.8
[09:20:42,208][INFO][main][IgniteKernal] VM information: Java(TM) SE Runtime
Environment 1.8.0_20-b26 Oracle Corporation Java HotSpot(TM) Client VM
25.20-b23
[09:20:42,223][INFO][main][IgniteKernal] VM total memory: 0.24GB
[09:20:42,223][INFO][main][IgniteKernal] Remote Management [restart: on,
REST: on, JMX (remote: on, port: 49237, auth: off, ssl: off)]
[09:20:42,223][INFO][main][IgniteKernal]
IGNITE_HOME=C:\projects\Ignite\apache-ignite-fabric-1.7.0-bin
[09:20:42,223][INFO][main][IgniteKernal] VM arguments: [-DIGNITE_QUIET=true,
-DIGNITE_SUCCESS_FILE=C:\projects\Ignite\apache-ignite-fabric-1.7.0-bin\work\ignite_success_060af0bc-5e6b-4a1b-ab1e-175170bd5eae,
-Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.port=49237,
-Dcom.sun.management.jmxremote.authenticate=false,
-Dcom.sun.management.jmxremote.ssl=false,
-DIGNITE_HOME=C:\projects\Ignite\apache-ignite-fabric-1.7.0-bin,
-DIGNITE_PROG_NAME=ignite.bat]
[09:20:42,223][INFO][main][IgniteKernal] Configured caches
['ignite-marshaller-sys-cache', 'ignite-sys-cache',
'ignite-atomics-sys-cache']
[09:20:42,223][INFO][main][IgniteKernal] 3-rd party licenses can be found
at: C:\projects\Ignite\apache-ignite-fabric-1.7.0-bin\libs\licenses
[09:20:42,239][WARNING][pub-#3%null%][GridDiagnostic] Initial heap size is
16MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[09:20:45,297][INFO][main][IgniteKernal] Non-loopback local IPs:
10.204.48.143, 192.168.1.3
and I am using log4j-2 configuration, copied log4j2 lib to lib folder ? what
am I missing here ?
Thanks for all your help..
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11125.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by Andrey Gura <ag...@apache.org>.
Rishi,
I think that log aggreagtion task can't be solved only by log4j
configuration. But you can reduce logs size and amount of log files
using different log level.
Also if your application offen log something you can do it using
GridLogThrottle class (or LT alias).
On Fri, Mar 10, 2017 at 6:00 PM, Rishi Yagnik <ri...@gmail.com> wrote:
> Andre,
>
> I am asking is there a way to aggregate log file with log4j
> /log4j-2configuration, currently I believe it s creating log file per
> session which produces 30-50k log files.
>
> This also creates problem in debugging when something gets wrong which log
> files to see since it s generating log file in humongous amount.
>
> Let us stick to Ignite here I know there are many tools out there which can
> do the job for me, I see it s a problem when you are monitoring ignite
> every 3 min to get different statistic, it is hard to get into root of the
> problem due to log file generated for every session.
>
> Any thoughts further ..
>
>
>
> On Fri, Mar 10, 2017 at 8:54 AM, Andrey Gura <ag...@apache.org> wrote:
>>
>> There are many ways to aggregate logs but it is all about special
>> instruments like LogStash, Kibana, etc, not about Ignite.
>>
>> I'm sure you will find a lot of information in your favorite search
>> engine.
>>
>> On Fri, Mar 10, 2017 at 5:17 PM, ignite_user2016 <ri...@gmail.com>
>> wrote:
>> > Any reply would be helpful here..
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11116.html
>> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>
>
>
> --
> Rishi Yagnik
Re: Aggregation of log files
Posted by Rishi Yagnik <ri...@gmail.com>.
Andre,
I am asking is there a way to aggregate log file with log4j
/log4j-2configuration, currently I believe it s creating log file per
session which produces 30-50k log files.
This also creates problem in debugging when something gets wrong which log
files to see since it s generating log file in humongous amount.
Let us stick to Ignite here I know there are many tools out there which can
do the job for me, I see it s a problem when you are monitoring ignite
every 3 min to get different statistic, it is hard to get into root of the
problem due to log file generated for every session.
Any thoughts further ..
On Fri, Mar 10, 2017 at 8:54 AM, Andrey Gura <ag...@apache.org> wrote:
> There are many ways to aggregate logs but it is all about special
> instruments like LogStash, Kibana, etc, not about Ignite.
>
> I'm sure you will find a lot of information in your favorite search engine.
>
> On Fri, Mar 10, 2017 at 5:17 PM, ignite_user2016 <ri...@gmail.com>
> wrote:
> > Any reply would be helpful here..
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11116.html
> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Rishi Yagnik
Re: Aggregation of log files
Posted by Andrey Gura <ag...@apache.org>.
There are many ways to aggregate logs but it is all about special
instruments like LogStash, Kibana, etc, not about Ignite.
I'm sure you will find a lot of information in your favorite search engine.
On Fri, Mar 10, 2017 at 5:17 PM, ignite_user2016 <ri...@gmail.com> wrote:
> Any reply would be helpful here..
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11116.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
Any reply would be helpful here..
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11116.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Aggregation of log files
Posted by ignite_user2016 <ri...@gmail.com>.
I found my answers I think it is working by design so user always has a
choice to override logger with jcl logger.
Thank you for all your help,
Rishi
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Aggregation-of-log-files-tp11102p11140.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.