You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Amit Oberoi <ao...@TechMahindra.com> on 2011/07/07 17:14:04 UTC
Issue with IDField in CustomSQLDBReceiver
All,
I am using the latest developer snapshot 2.0.1 for Chainsaw V2. I have
configured CustomSQLDBReceiver with My SQL server.
My problem is every time the receiver refreshes it gets all the logs in
the database instead of the new ones. Probably it is ignoring the
IDField param for SQL.
It is apparently looking at the IDField because it complains if I don't
set it or have it set to a field that is not a number; however, it is
not using it in the sql.
Can anyone help in sorting this out; I'll be happy to debug and work on
a patch in case someone points me to the code.
Below is my table schema and receiver configuration.
Table Schema:
+------------------+-----------------+---------+---------+--------------
-----------------------+---------------------------------------------+
| Field | Type | Null | Key
| Default | Extra
|
+------------------+-----------------+---------+---------+--------------
-----------------------+---------------------------------------------+
| COUNTER | int(11) | NO | PRI | NULL
| auto_increment |
| TIMESTAMP | timestamp | NO | |
CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| THREAD | varchar(255) | YES | | NULL
|
|
| CLASS | varchar(255) | YES | | NULL
|
|
| METHOD | varchar(100) | YES | | NULL
|
|
| FILE | varchar(100) | YES | |
NULL |
|
| LINE | int(11) | YES |
| NULL |
|
| PRIORITY | varchar(50) | YES | | NULL
|
|
| LOGGER | varchar(255) | NO | | NULL
|
|
| MESSAGE | varchar(1000) | NO | | NULL
|
|
| NDC | varchar(255) | YES | |
NULL |
|
| MDC | varchar(255) | YES | | NULL
|
|
+------------------+-----------------+---------+---------+--------------
-----------------------+---------------------------------------------+
Receiver Configuration:
<plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="Receiver">
<param name="active" value="true" />
<connectionSource
class="org.apache.log4j.db.DriverManagerConnectionSource">
<param name="user" value="user1"/>
<param name="password" value="passwd"/>
<param name="driverClass" value="org.gjt.mm.mysql.Driver"/>
<param name="url"
value="jdbc:mysql://10.2.21.127:3306/Logs"/>
</connectionSource>
<param name="refreshMillis" value="5000"/>
<param name="sql" value='select logger as LOGGER, timestamp as
TIMESTAMP, priority as LEVEL, thread as THREAD, message as MESSAGE, ndc
as NDC, mdc as MDC, class as CLASS, method as METHOD, file as FILE, line
as LINE, concat("{{application,databaselogs,hostname,mymachine,
log4jid,", COUNTER,"}}") as PROPERTIES, "" as THROWABLE from logs'/>
<param name="IDField" value="COUNTER"/>
<param name="threshold" value="ALL" />
</plugin>
Regards
Amit Oberoi
============================================================================================================================Disclaimer: This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at <a href="http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.com/Disclaimer.html</a> externally and <a href="http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech Mahindra.============================================================================================================================
RE: Issue with IDField in CustomSQLDBReceiver
Posted by Amit Oberoi <ao...@TechMahindra.com>.
Many Thanks Scott.
Guess I have figured out the solution. The problem was with an extra
space for log4jid token within PROERTIES. I have added a .trim() to
remove it and now it works fine for me. Probably you would like to do
the changes in repository as well, if{} block at line 389 in file
CustomSQLDBReceiver.java should look like below. I am attaching the
updated source for CustomSQLDBReceiver.
******************************************
389 if (tokenName.trim().equals(LOG4J_ID_KEY)) { //
Note the .trim() added to tokenName
******************************************
390 try {
391 int thisInt =
Integer.parseInt(value);
392 value =
String.valueOf(thisInt);
393 if (thisInt > lastID) {
394 lastID = thisInt;
395 }
396 } catch (Exception e) {
397 }
398 }
-----Original Message-----
From: Scott Deboy [mailto:scott.deboy@gmail.com]
Sent: Friday, July 08, 2011 7:41 AM
To: Log4J Users List
Subject: Re: Issue with IDField in CustomSQLDBReceiver
All of the code is available in subversion...to build the
CustomSQLDBReceiver you can download the receivers companion and build
it
with maven.
I was running from source myself - I'll try the DMG from the developer
snapshot.
Here is the repo information for the receivers companion..
http://logging.apache.org/log4j/companions/receivers/source-repository.h
tml
Scott
On Thu, Jul 7, 2011 at 6:38 PM, Amit Oberoi
<ao...@techmahindra.com>wrote:
> Scott,
>
> I don't find any difference in mine and yours configuration; in fact I
> have taken it from one of your replies to my earlier emails.
>
> Is there anywhere I can get the code so that I can debug.
>
> Regards
>
> Amit Oberoi
>
> -----Original Message-----
> From: Scott Deboy [mailto:scott.deboy@gmail.com]
> Sent: Friday, July 08, 2011 12:24 AM
> To: Log4J Users List
> Subject: Re: Issue with IDField in CustomSQLDBReceiver
>
> Yes, IDField is used to dynamically update the SQL statement with a
> 'where'
> clause: (idfield) > (last known IDField value), essentially allowing
it
> to
> 'tail' the table contents.
>
> Here is the config I was able to use to get it to tail with a very
> simple
> mysql db (verified by printing out the sql statement in the
receiver..I
> saw
> the where clause modified), and the new results were added to the
table.
>
> <?xml version="1.0" encoding="UTF-8"?>
> <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"
> debug="true">
> <plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="DB">
> <connectionSource
> class="org.apache.log4j.db.DriverManagerConnectionSource">
> <param name="user" value="someusername"/>
> <param name="password" value="someuserpassword"/>
> <param name="driverClass" value="com.mysql.jdbc.Driver"/>
> <param name="url" value="jdbc:mysql://127.0.0.1:3306/test"/>
> </connectionSource>
> <param name="IDField" value="COUNTER"/>
> <param name="class" value="class
> org.apache.log4j.db.CustomSQLDBReceiver"/>
> <param name="name" value="DB"/>
> <param name="paused" value="false"/>
> <param name="refreshMillis" value="1000"/>
> <param name="sql" value="select logger as LOGGER,
> CURRENT_TIMESTAMP
> as TIMESTAMP, "" as LEVEL, "" as THREAD, message
as
> MESSAGE, "" as NDC, "" as MDC, "" as
> CLASS,
> "" as METHOD, "" as FILE, "" as LINE,
> concat("{{log4jid,", COUNTER,"}}") as PROPERTIES,
> "" as THROWABLE from logging"/>
> </plugin>
> </log4j:configuration>
>
> Here is the very-basic schema I used to test:
>
> Field Type Null Key Default Extra
> COUNTER int(11) NO PRI NULL auto_increment
> MESSAGE varchar(1000) YES NULL
> LOGGER varchar(255) YES NULL
>
> Scott
>
> On Thu, Jul 7, 2011 at 8:14 AM, Amit Oberoi
> <ao...@techmahindra.com>wrote:
>
> > All,
> >
> >
> >
> > I am using the latest developer snapshot 2.0.1 for Chainsaw V2. I
have
> > configured CustomSQLDBReceiver with My SQL server.
> >
> >
> >
> > My problem is every time the receiver refreshes it gets all the logs
> in
> > the database instead of the new ones. Probably it is ignoring the
> > IDField param for SQL.
> >
> >
> >
> > It is apparently looking at the IDField because it complains if I
> don't
> > set it or have it set to a field that is not a number; however, it
is
> > not using it in the sql.
> >
> >
> >
> > Can anyone help in sorting this out; I'll be happy to debug and work
> on
> > a patch in case someone points me to the code.
> >
> >
> >
> > Below is my table schema and receiver configuration.
> >
> >
> >
> > Table Schema:
> >
> >
> >
> >
>
+------------------+-----------------+---------+---------+--------------
> >
-----------------------+---------------------------------------------+
> >
> > | Field | Type | Null | Key
> > | Default | Extra
> > |
> >
> >
>
+------------------+-----------------+---------+---------+--------------
> >
-----------------------+---------------------------------------------+
> >
> > | COUNTER | int(11) | NO | PRI |
NULL
> > | auto_increment |
> >
> > | TIMESTAMP | timestamp | NO | |
> > CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP
|
> >
> > | THREAD | varchar(255) | YES | | NULL
> > |
> > |
> >
> > | CLASS | varchar(255) | YES | |
> NULL
> > |
> > |
> >
> > | METHOD | varchar(100) | YES | | NULL
> > |
> > |
> >
> > | FILE | varchar(100) | YES |
|
> > NULL |
> > |
> >
> > | LINE | int(11) | YES |
> > | NULL |
> > |
> >
> > | PRIORITY | varchar(50) | YES | |
> NULL
> > |
> > |
> >
> > | LOGGER | varchar(255) | NO | | NULL
> > |
> > |
> >
> > | MESSAGE | varchar(1000) | NO | | NULL
> > |
> > |
> >
> > | NDC | varchar(255) | YES | |
> > NULL |
> > |
> >
> > | MDC | varchar(255) | YES | |
> NULL
> > |
> > |
> >
> >
>
+------------------+-----------------+---------+---------+--------------
> >
-----------------------+---------------------------------------------+
> >
> >
> >
> > Receiver Configuration:
> >
> >
> >
> > <plugin class="org.apache.log4j.db.CustomSQLDBReceiver"
> name="Receiver">
> >
> > <param name="active" value="true" />
> >
> > <connectionSource
> > class="org.apache.log4j.db.DriverManagerConnectionSource">
> >
> > <param name="user" value="user1"/>
> >
> > <param name="password" value="passwd"/>
> >
> > <param name="driverClass"
value="org.gjt.mm.mysql.Driver"/>
> >
> > <param name="url"
> > value="jdbc:mysql://10.2.21.127:3306/Logs"/>
> >
> > </connectionSource>
> >
> > <param name="refreshMillis" value="5000"/>
> >
> > <param name="sql" value='select logger as LOGGER, timestamp as
> > TIMESTAMP, priority as LEVEL, thread as THREAD, message as MESSAGE,
> ndc
> > as NDC, mdc as MDC, class as CLASS, method as METHOD, file as FILE,
> line
> > as LINE, concat("{{application,databaselogs,hostname,mymachine,
> > log4jid,", COUNTER,"}}") as PROPERTIES, "" as THROWABLE from logs'/>
> >
> > <param name="IDField" value="COUNTER"/>
> >
> > <param name="threshold" value="ALL" />
> >
> > </plugin>
> >
> >
> >
> >
> >
> >
> >
> > Regards
> >
> >
> >
> > Amit Oberoi
> >
> >
> >
> >
> >
>
========================================================================
> ====================================================Disclaimer:
> > This message and the information contained herein is proprietary
and
> > confidential and subject to the Tech Mahindra policy statement, you
> may
> > review the policy at <a
> href="http://www.techmahindra.com/Disclaimer.html
> > ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a
> href="
> > http://tim.techmahindra.com/Disclaimer.html">
> > http://tim.techmahindra.com/Disclaimer.html</a> internally within
Tech
> >
>
Mahindra.===============================================================
> =============================================================
> >
>
>
========================================================================
====================================================Disclaimer:
> This message and the information contained herein is proprietary and
> confidential and subject to the Tech Mahindra policy statement, you
may
> review the policy at <a
href="http://www.techmahindra.com/Disclaimer.html
> ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a
href="
> http://tim.techmahindra.com/Disclaimer.html">
> http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech
>
Mahindra.===============================================================
=============================================================
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
============================================================================================================================Disclaimer: This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at <a href="http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.com/Disclaimer.html</a> externally and <a href="http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech Mahindra.============================================================================================================================
Re: Issue with IDField in CustomSQLDBReceiver
Posted by Scott Deboy <sc...@gmail.com>.
All of the code is available in subversion...to build the
CustomSQLDBReceiver you can download the receivers companion and build it
with maven.
I was running from source myself - I'll try the DMG from the developer
snapshot.
Here is the repo information for the receivers companion..
http://logging.apache.org/log4j/companions/receivers/source-repository.html
Scott
On Thu, Jul 7, 2011 at 6:38 PM, Amit Oberoi <ao...@techmahindra.com>wrote:
> Scott,
>
> I don't find any difference in mine and yours configuration; in fact I
> have taken it from one of your replies to my earlier emails.
>
> Is there anywhere I can get the code so that I can debug.
>
> Regards
>
> Amit Oberoi
>
> -----Original Message-----
> From: Scott Deboy [mailto:scott.deboy@gmail.com]
> Sent: Friday, July 08, 2011 12:24 AM
> To: Log4J Users List
> Subject: Re: Issue with IDField in CustomSQLDBReceiver
>
> Yes, IDField is used to dynamically update the SQL statement with a
> 'where'
> clause: (idfield) > (last known IDField value), essentially allowing it
> to
> 'tail' the table contents.
>
> Here is the config I was able to use to get it to tail with a very
> simple
> mysql db (verified by printing out the sql statement in the receiver..I
> saw
> the where clause modified), and the new results were added to the table.
>
> <?xml version="1.0" encoding="UTF-8"?>
> <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"
> debug="true">
> <plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="DB">
> <connectionSource
> class="org.apache.log4j.db.DriverManagerConnectionSource">
> <param name="user" value="someusername"/>
> <param name="password" value="someuserpassword"/>
> <param name="driverClass" value="com.mysql.jdbc.Driver"/>
> <param name="url" value="jdbc:mysql://127.0.0.1:3306/test"/>
> </connectionSource>
> <param name="IDField" value="COUNTER"/>
> <param name="class" value="class
> org.apache.log4j.db.CustomSQLDBReceiver"/>
> <param name="name" value="DB"/>
> <param name="paused" value="false"/>
> <param name="refreshMillis" value="1000"/>
> <param name="sql" value="select logger as LOGGER,
> CURRENT_TIMESTAMP
> as TIMESTAMP, "" as LEVEL, "" as THREAD, message as
> MESSAGE, "" as NDC, "" as MDC, "" as
> CLASS,
> "" as METHOD, "" as FILE, "" as LINE,
> concat("{{log4jid,", COUNTER,"}}") as PROPERTIES,
> "" as THROWABLE from logging"/>
> </plugin>
> </log4j:configuration>
>
> Here is the very-basic schema I used to test:
>
> Field Type Null Key Default Extra
> COUNTER int(11) NO PRI NULL auto_increment
> MESSAGE varchar(1000) YES NULL
> LOGGER varchar(255) YES NULL
>
> Scott
>
> On Thu, Jul 7, 2011 at 8:14 AM, Amit Oberoi
> <ao...@techmahindra.com>wrote:
>
> > All,
> >
> >
> >
> > I am using the latest developer snapshot 2.0.1 for Chainsaw V2. I have
> > configured CustomSQLDBReceiver with My SQL server.
> >
> >
> >
> > My problem is every time the receiver refreshes it gets all the logs
> in
> > the database instead of the new ones. Probably it is ignoring the
> > IDField param for SQL.
> >
> >
> >
> > It is apparently looking at the IDField because it complains if I
> don't
> > set it or have it set to a field that is not a number; however, it is
> > not using it in the sql.
> >
> >
> >
> > Can anyone help in sorting this out; I'll be happy to debug and work
> on
> > a patch in case someone points me to the code.
> >
> >
> >
> > Below is my table schema and receiver configuration.
> >
> >
> >
> > Table Schema:
> >
> >
> >
> >
> +------------------+-----------------+---------+---------+--------------
> > -----------------------+---------------------------------------------+
> >
> > | Field | Type | Null | Key
> > | Default | Extra
> > |
> >
> >
> +------------------+-----------------+---------+---------+--------------
> > -----------------------+---------------------------------------------+
> >
> > | COUNTER | int(11) | NO | PRI | NULL
> > | auto_increment |
> >
> > | TIMESTAMP | timestamp | NO | |
> > CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
> >
> > | THREAD | varchar(255) | YES | | NULL
> > |
> > |
> >
> > | CLASS | varchar(255) | YES | |
> NULL
> > |
> > |
> >
> > | METHOD | varchar(100) | YES | | NULL
> > |
> > |
> >
> > | FILE | varchar(100) | YES | |
> > NULL |
> > |
> >
> > | LINE | int(11) | YES |
> > | NULL |
> > |
> >
> > | PRIORITY | varchar(50) | YES | |
> NULL
> > |
> > |
> >
> > | LOGGER | varchar(255) | NO | | NULL
> > |
> > |
> >
> > | MESSAGE | varchar(1000) | NO | | NULL
> > |
> > |
> >
> > | NDC | varchar(255) | YES | |
> > NULL |
> > |
> >
> > | MDC | varchar(255) | YES | |
> NULL
> > |
> > |
> >
> >
> +------------------+-----------------+---------+---------+--------------
> > -----------------------+---------------------------------------------+
> >
> >
> >
> > Receiver Configuration:
> >
> >
> >
> > <plugin class="org.apache.log4j.db.CustomSQLDBReceiver"
> name="Receiver">
> >
> > <param name="active" value="true" />
> >
> > <connectionSource
> > class="org.apache.log4j.db.DriverManagerConnectionSource">
> >
> > <param name="user" value="user1"/>
> >
> > <param name="password" value="passwd"/>
> >
> > <param name="driverClass" value="org.gjt.mm.mysql.Driver"/>
> >
> > <param name="url"
> > value="jdbc:mysql://10.2.21.127:3306/Logs"/>
> >
> > </connectionSource>
> >
> > <param name="refreshMillis" value="5000"/>
> >
> > <param name="sql" value='select logger as LOGGER, timestamp as
> > TIMESTAMP, priority as LEVEL, thread as THREAD, message as MESSAGE,
> ndc
> > as NDC, mdc as MDC, class as CLASS, method as METHOD, file as FILE,
> line
> > as LINE, concat("{{application,databaselogs,hostname,mymachine,
> > log4jid,", COUNTER,"}}") as PROPERTIES, "" as THROWABLE from logs'/>
> >
> > <param name="IDField" value="COUNTER"/>
> >
> > <param name="threshold" value="ALL" />
> >
> > </plugin>
> >
> >
> >
> >
> >
> >
> >
> > Regards
> >
> >
> >
> > Amit Oberoi
> >
> >
> >
> >
> >
> ========================================================================
> ====================================================Disclaimer:
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Tech Mahindra policy statement, you
> may
> > review the policy at <a
> href="http://www.techmahindra.com/Disclaimer.html
> > ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a
> href="
> > http://tim.techmahindra.com/Disclaimer.html">
> > http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech
> >
> Mahindra.===============================================================
> =============================================================
> >
>
> ============================================================================================================================Disclaimer:
> This message and the information contained herein is proprietary and
> confidential and subject to the Tech Mahindra policy statement, you may
> review the policy at <a href="http://www.techmahindra.com/Disclaimer.html
> ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a href="
> http://tim.techmahindra.com/Disclaimer.html">
> http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech
> Mahindra.============================================================================================================================
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
RE: Issue with IDField in CustomSQLDBReceiver
Posted by Amit Oberoi <ao...@TechMahindra.com>.
Scott,
I don't find any difference in mine and yours configuration; in fact I
have taken it from one of your replies to my earlier emails.
Is there anywhere I can get the code so that I can debug.
Regards
Amit Oberoi
-----Original Message-----
From: Scott Deboy [mailto:scott.deboy@gmail.com]
Sent: Friday, July 08, 2011 12:24 AM
To: Log4J Users List
Subject: Re: Issue with IDField in CustomSQLDBReceiver
Yes, IDField is used to dynamically update the SQL statement with a
'where'
clause: (idfield) > (last known IDField value), essentially allowing it
to
'tail' the table contents.
Here is the config I was able to use to get it to tail with a very
simple
mysql db (verified by printing out the sql statement in the receiver..I
saw
the where clause modified), and the new results were added to the table.
<?xml version="1.0" encoding="UTF-8"?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"
debug="true">
<plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="DB">
<connectionSource
class="org.apache.log4j.db.DriverManagerConnectionSource">
<param name="user" value="someusername"/>
<param name="password" value="someuserpassword"/>
<param name="driverClass" value="com.mysql.jdbc.Driver"/>
<param name="url" value="jdbc:mysql://127.0.0.1:3306/test"/>
</connectionSource>
<param name="IDField" value="COUNTER"/>
<param name="class" value="class
org.apache.log4j.db.CustomSQLDBReceiver"/>
<param name="name" value="DB"/>
<param name="paused" value="false"/>
<param name="refreshMillis" value="1000"/>
<param name="sql" value="select logger as LOGGER,
CURRENT_TIMESTAMP
as TIMESTAMP, "" as LEVEL, "" as THREAD, message as
MESSAGE, "" as NDC, "" as MDC, "" as
CLASS,
"" as METHOD, "" as FILE, "" as LINE,
concat("{{log4jid,", COUNTER,"}}") as PROPERTIES,
"" as THROWABLE from logging"/>
</plugin>
</log4j:configuration>
Here is the very-basic schema I used to test:
Field Type Null Key Default Extra
COUNTER int(11) NO PRI NULL auto_increment
MESSAGE varchar(1000) YES NULL
LOGGER varchar(255) YES NULL
Scott
On Thu, Jul 7, 2011 at 8:14 AM, Amit Oberoi
<ao...@techmahindra.com>wrote:
> All,
>
>
>
> I am using the latest developer snapshot 2.0.1 for Chainsaw V2. I have
> configured CustomSQLDBReceiver with My SQL server.
>
>
>
> My problem is every time the receiver refreshes it gets all the logs
in
> the database instead of the new ones. Probably it is ignoring the
> IDField param for SQL.
>
>
>
> It is apparently looking at the IDField because it complains if I
don't
> set it or have it set to a field that is not a number; however, it is
> not using it in the sql.
>
>
>
> Can anyone help in sorting this out; I'll be happy to debug and work
on
> a patch in case someone points me to the code.
>
>
>
> Below is my table schema and receiver configuration.
>
>
>
> Table Schema:
>
>
>
>
+------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
> | Field | Type | Null | Key
> | Default | Extra
> |
>
>
+------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
> | COUNTER | int(11) | NO | PRI | NULL
> | auto_increment |
>
> | TIMESTAMP | timestamp | NO | |
> CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
>
> | THREAD | varchar(255) | YES | | NULL
> |
> |
>
> | CLASS | varchar(255) | YES | |
NULL
> |
> |
>
> | METHOD | varchar(100) | YES | | NULL
> |
> |
>
> | FILE | varchar(100) | YES | |
> NULL |
> |
>
> | LINE | int(11) | YES |
> | NULL |
> |
>
> | PRIORITY | varchar(50) | YES | |
NULL
> |
> |
>
> | LOGGER | varchar(255) | NO | | NULL
> |
> |
>
> | MESSAGE | varchar(1000) | NO | | NULL
> |
> |
>
> | NDC | varchar(255) | YES | |
> NULL |
> |
>
> | MDC | varchar(255) | YES | |
NULL
> |
> |
>
>
+------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
>
>
> Receiver Configuration:
>
>
>
> <plugin class="org.apache.log4j.db.CustomSQLDBReceiver"
name="Receiver">
>
> <param name="active" value="true" />
>
> <connectionSource
> class="org.apache.log4j.db.DriverManagerConnectionSource">
>
> <param name="user" value="user1"/>
>
> <param name="password" value="passwd"/>
>
> <param name="driverClass" value="org.gjt.mm.mysql.Driver"/>
>
> <param name="url"
> value="jdbc:mysql://10.2.21.127:3306/Logs"/>
>
> </connectionSource>
>
> <param name="refreshMillis" value="5000"/>
>
> <param name="sql" value='select logger as LOGGER, timestamp as
> TIMESTAMP, priority as LEVEL, thread as THREAD, message as MESSAGE,
ndc
> as NDC, mdc as MDC, class as CLASS, method as METHOD, file as FILE,
line
> as LINE, concat("{{application,databaselogs,hostname,mymachine,
> log4jid,", COUNTER,"}}") as PROPERTIES, "" as THROWABLE from logs'/>
>
> <param name="IDField" value="COUNTER"/>
>
> <param name="threshold" value="ALL" />
>
> </plugin>
>
>
>
>
>
>
>
> Regards
>
>
>
> Amit Oberoi
>
>
>
>
>
========================================================================
====================================================Disclaimer:
> This message and the information contained herein is proprietary and
> confidential and subject to the Tech Mahindra policy statement, you
may
> review the policy at <a
href="http://www.techmahindra.com/Disclaimer.html
> ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a
href="
> http://tim.techmahindra.com/Disclaimer.html">
> http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech
>
Mahindra.===============================================================
=============================================================
>
============================================================================================================================Disclaimer: This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at <a href="http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.com/Disclaimer.html</a> externally and <a href="http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech Mahindra.============================================================================================================================
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Issue with IDField in CustomSQLDBReceiver
Posted by Scott Deboy <sc...@gmail.com>.
Yes, IDField is used to dynamically update the SQL statement with a 'where'
clause: (idfield) > (last known IDField value), essentially allowing it to
'tail' the table contents.
Here is the config I was able to use to get it to tail with a very simple
mysql db (verified by printing out the sql statement in the receiver..I saw
the where clause modified), and the new results were added to the table.
<?xml version="1.0" encoding="UTF-8"?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"
debug="true">
<plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="DB">
<connectionSource
class="org.apache.log4j.db.DriverManagerConnectionSource">
<param name="user" value="someusername"/>
<param name="password" value="someuserpassword"/>
<param name="driverClass" value="com.mysql.jdbc.Driver"/>
<param name="url" value="jdbc:mysql://127.0.0.1:3306/test"/>
</connectionSource>
<param name="IDField" value="COUNTER"/>
<param name="class" value="class
org.apache.log4j.db.CustomSQLDBReceiver"/>
<param name="name" value="DB"/>
<param name="paused" value="false"/>
<param name="refreshMillis" value="1000"/>
<param name="sql" value="select logger as LOGGER, CURRENT_TIMESTAMP
as TIMESTAMP, "" as LEVEL, "" as THREAD, message as
MESSAGE, "" as NDC, "" as MDC, "" as CLASS,
"" as METHOD, "" as FILE, "" as LINE,
concat("{{log4jid,", COUNTER,"}}") as PROPERTIES,
"" as THROWABLE from logging"/>
</plugin>
</log4j:configuration>
Here is the very-basic schema I used to test:
Field Type Null Key Default Extra
COUNTER int(11) NO PRI NULL auto_increment
MESSAGE varchar(1000) YES NULL
LOGGER varchar(255) YES NULL
Scott
On Thu, Jul 7, 2011 at 8:14 AM, Amit Oberoi <ao...@techmahindra.com>wrote:
> All,
>
>
>
> I am using the latest developer snapshot 2.0.1 for Chainsaw V2. I have
> configured CustomSQLDBReceiver with My SQL server.
>
>
>
> My problem is every time the receiver refreshes it gets all the logs in
> the database instead of the new ones. Probably it is ignoring the
> IDField param for SQL.
>
>
>
> It is apparently looking at the IDField because it complains if I don't
> set it or have it set to a field that is not a number; however, it is
> not using it in the sql.
>
>
>
> Can anyone help in sorting this out; I'll be happy to debug and work on
> a patch in case someone points me to the code.
>
>
>
> Below is my table schema and receiver configuration.
>
>
>
> Table Schema:
>
>
>
> +------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
> | Field | Type | Null | Key
> | Default | Extra
> |
>
> +------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
> | COUNTER | int(11) | NO | PRI | NULL
> | auto_increment |
>
> | TIMESTAMP | timestamp | NO | |
> CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
>
> | THREAD | varchar(255) | YES | | NULL
> |
> |
>
> | CLASS | varchar(255) | YES | | NULL
> |
> |
>
> | METHOD | varchar(100) | YES | | NULL
> |
> |
>
> | FILE | varchar(100) | YES | |
> NULL |
> |
>
> | LINE | int(11) | YES |
> | NULL |
> |
>
> | PRIORITY | varchar(50) | YES | | NULL
> |
> |
>
> | LOGGER | varchar(255) | NO | | NULL
> |
> |
>
> | MESSAGE | varchar(1000) | NO | | NULL
> |
> |
>
> | NDC | varchar(255) | YES | |
> NULL |
> |
>
> | MDC | varchar(255) | YES | | NULL
> |
> |
>
> +------------------+-----------------+---------+---------+--------------
> -----------------------+---------------------------------------------+
>
>
>
> Receiver Configuration:
>
>
>
> <plugin class="org.apache.log4j.db.CustomSQLDBReceiver" name="Receiver">
>
> <param name="active" value="true" />
>
> <connectionSource
> class="org.apache.log4j.db.DriverManagerConnectionSource">
>
> <param name="user" value="user1"/>
>
> <param name="password" value="passwd"/>
>
> <param name="driverClass" value="org.gjt.mm.mysql.Driver"/>
>
> <param name="url"
> value="jdbc:mysql://10.2.21.127:3306/Logs"/>
>
> </connectionSource>
>
> <param name="refreshMillis" value="5000"/>
>
> <param name="sql" value='select logger as LOGGER, timestamp as
> TIMESTAMP, priority as LEVEL, thread as THREAD, message as MESSAGE, ndc
> as NDC, mdc as MDC, class as CLASS, method as METHOD, file as FILE, line
> as LINE, concat("{{application,databaselogs,hostname,mymachine,
> log4jid,", COUNTER,"}}") as PROPERTIES, "" as THROWABLE from logs'/>
>
> <param name="IDField" value="COUNTER"/>
>
> <param name="threshold" value="ALL" />
>
> </plugin>
>
>
>
>
>
>
>
> Regards
>
>
>
> Amit Oberoi
>
>
>
>
> ============================================================================================================================Disclaimer:
> This message and the information contained herein is proprietary and
> confidential and subject to the Tech Mahindra policy statement, you may
> review the policy at <a href="http://www.techmahindra.com/Disclaimer.html
> ">http://www.techmahindra.com/Disclaimer.html</a> externally and <a href="
> http://tim.techmahindra.com/Disclaimer.html">
> http://tim.techmahindra.com/Disclaimer.html</a> internally within Tech
> Mahindra.============================================================================================================================
>