You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by liyuj <gi...@git.apache.org> on 2017/06/01 11:16:51 UTC
[GitHub] logging-log4j2 pull request #82: Add a new LuceneAppender which writes loggi...
GitHub user liyuj opened a pull request:
https://github.com/apache/logging-log4j2/pull/82
Add a new LuceneAppender which writes logging events to a lucene index library.
Add a new LuceneAppender which writes logging events to a lucene index library.The log4j2.xml configuration is as follows:
<Lucene name="lucene" ignoreExceptions="true" target="/target/lucene/index" expiryTime=“1296000”>
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %class{36} %L %M - %msg%xEx%n"/>
<IndexField name="time" pattern="%d{UNIX_MILLIS}" type="LongField"/>
<IndexField name="level" pattern="%-5level" />
<IndexField name="content" pattern="%d{HH:mm:ss.SSS} %-5level %class{36} %L %M - %msg%xEx%n"/>
</Lucene>
this appender relies on the Lucene 5.5.4 version.
this patch adds the corresponding test cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liyuj/logging-log4j2 master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/logging-log4j2/pull/82.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #82
----
commit 4b85794d18a6b80974313c6c6d47ff9b04b6294d
Author: liyuj <18...@163.com>
Date: 2017-06-01T11:10:36Z
Add a new LuceneAppender which writes logging events to a lucene index
library.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Hi,
I can't recall if we have one but I brought in the code locally and started
fiddling with it.
Please see the Lucene 5 branch here:
https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;a=shortlog;h=refs/heads/Lucene5
Gary
On Fri, Jul 7, 2017 at 1:49 PM, Mikael Ståldal <no...@github.com>
wrote:
> Which JIRA issue is this?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/logging-log4j2/pull/82#issuecomment-313789609>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABIfN0Al2drnd0RlBqO2T2r9t2koHkkQks5sLpnwgaJpZM4Ns1oB>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
What is the current state of this PR?
If want to continue, what do I need to do?
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
I can provide, but what does it contain? In what format?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which
writes logging even...
Posted by kenneth mcfarland <ke...@gmail.com>.
Forgive me, I just mean the close() method.
On Mon, Jul 24, 2017 at 8:30 PM, kenneth mcfarland <
kennethpmcfarland@gmail.com> wrote:
> Why don't you call close() or commit() on the indexwriter? If log4j2 is
> serious about being an auditing platform some attention to flushing and
> such should be addressed. Note I am not an expert but I believe I am
> somewhere in the ballpark.
>
> On Mon, Jul 24, 2017 at 8:27 PM, garydgregory <gi...@git.apache.org> wrote:
>
>> Github user garydgregory commented on the issue:
>>
>> https://github.com/apache/logging-log4j2/pull/82
>>
>> Thank you for your submission @liyuj ! Can you try the "Lucene5"
>> branch and confirm that it works for you?
>>
>>
>> ---
>> If your project is set up for it, you can reply to this email and have
>> your
>> reply appear on GitHub as well. If your project does not have this feature
>> enabled and wishes so, or if the feature is enabled but not working,
>> please
>> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
>> with INFRA.
>> ---
>>
>
>
Re: [GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which
writes logging even...
Posted by kenneth mcfarland <ke...@gmail.com>.
Why don't you call close() or commit() on the indexwriter? If log4j2 is
serious about being an auditing platform some attention to flushing and
such should be addressed. Note I am not an expert but I believe I am
somewhere in the ballpark.
On Mon, Jul 24, 2017 at 8:27 PM, garydgregory <gi...@git.apache.org> wrote:
> Github user garydgregory commented on the issue:
>
> https://github.com/apache/logging-log4j2/pull/82
>
> Thank you for your submission @liyuj ! Can you try the "Lucene5"
> branch and confirm that it works for you?
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Thank you for your submission @liyuj ! Can you try the "Lucene5" branch and confirm that it works for you?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which
writes logging even...
Posted by Gary Gregory <ga...@gmail.com>.
I think that would be too expensive to do on each event.
Note that there are three levels: addDocument(), flush(), and commit(). I
could see that how often to flush and or commit should be configurable.
Either every N events, or every N TimeUnits.
Gary
On Mon, Jul 24, 2017 at 9:04 PM, kenneth mcfarland <
kennethpmcfarland@gmail.com> wrote:
> Ok I'm blind but maybe can add a commit() after line 193 right after
> addDocument() to be aggressive
>
> On Jul 24, 2017 8:54 PM, "garydgregory" <gi...@git.apache.org> wrote:
>
> > Github user garydgregory commented on the issue:
> >
> > https://github.com/apache/logging-log4j2/pull/82
> >
> > Is what is happening
> > in org.apache.logging.log4j.lucene5.appender.
> LuceneAppender.stop(long,
> > TimeUnit) not good enough?
> >
> > Gary
> >
> > On Mon, Jul 24, 2017 at 8:30 PM, asfbot <no...@github.com>
> > wrote:
> >
> > > kenneth mcfarland on dev@logging.apache.org replies:
> > > Why don't you call close() or commit() on the indexwriter? If
> log4j2
> > is
> > > serious about being an auditing platform some attention to flushing
> > and
> > > such should be addressed. Note I am not an expert but I believe I
> am
> > > somewhere in the ballpark.
> > >
> > > —
> > > You are receiving this because you commented.
> > > Reply to this email directly, view it on GitHub
> > > <https://github.com/apache/logging-log4j2/pull/82#
> > issuecomment-317618689>,
> > > or mute the thread
> > > <https://github.com/notifications/unsubscribe-auth/ABIfN6D_
> > jsLHaJif7ngIlu4q1HLTGYJdks5sRWFQgaJpZM4Ns1oB>
> > > .
> > >
> >
> >
> >
> > ---
> > If your project is set up for it, you can reply to this email and have
> your
> > reply appear on GitHub as well. If your project does not have this
> feature
> > enabled and wishes so, or if the feature is enabled but not working,
> please
> > contact infrastructure at infrastructure@apache.org or file a JIRA
> ticket
> > with INFRA.
> > ---
> >
>
Re: [GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which
writes logging even...
Posted by kenneth mcfarland <ke...@gmail.com>.
Ok I'm blind but maybe can add a commit() after line 193 right after
addDocument() to be aggressive
On Jul 24, 2017 8:54 PM, "garydgregory" <gi...@git.apache.org> wrote:
> Github user garydgregory commented on the issue:
>
> https://github.com/apache/logging-log4j2/pull/82
>
> Is what is happening
> in org.apache.logging.log4j.lucene5.appender.LuceneAppender.stop(long,
> TimeUnit) not good enough?
>
> Gary
>
> On Mon, Jul 24, 2017 at 8:30 PM, asfbot <no...@github.com>
> wrote:
>
> > kenneth mcfarland on dev@logging.apache.org replies:
> > Why don't you call close() or commit() on the indexwriter? If log4j2
> is
> > serious about being an auditing platform some attention to flushing
> and
> > such should be addressed. Note I am not an expert but I believe I am
> > somewhere in the ballpark.
> >
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub
> > <https://github.com/apache/logging-log4j2/pull/82#
> issuecomment-317618689>,
> > or mute the thread
> > <https://github.com/notifications/unsubscribe-auth/ABIfN6D_
> jsLHaJif7ngIlu4q1HLTGYJdks5sRWFQgaJpZM4Ns1oB>
> > .
> >
>
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Is what is happening
in org.apache.logging.log4j.lucene5.appender.LuceneAppender.stop(long,
TimeUnit) not good enough?
Gary
On Mon, Jul 24, 2017 at 8:30 PM, asfbot <no...@github.com> wrote:
> kenneth mcfarland on dev@logging.apache.org replies:
> Why don't you call close() or commit() on the indexwriter? If log4j2 is
> serious about being an auditing platform some attention to flushing and
> such should be addressed. Note I am not an expert but I believe I am
> somewhere in the ballpark.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/logging-log4j2/pull/82#issuecomment-317618689>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABIfN6D_jsLHaJif7ngIlu4q1HLTGYJdks5sRWFQgaJpZM4Ns1oB>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
I created a branch called "lucene5" in git. Can you test this branch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Can you provide any documentation for the new appender? That would help a LOT! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
This PR does not have a corresponding issue.
The code is used within our company and we think he has reusability, so it contributes to the community.
The functionality has been tested and no problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by mikaelstaldal <gi...@git.apache.org>.
Github user mikaelstaldal commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Which JIRA issue is this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Updating your PR to be based on the latest master would be great. In the master branch, we've divided plugins into more modules now, so there's more examples as well.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
I've created a new branch in our git repo called "Lucene5" based on your PR. There are some changes and fixes to allow the build to run. This WIP and not ready merge to master yet.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
https://www.zybuluo.com/liyuj/note/825932
I wrote a document. Can the community help me check?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
You can add docs to the [appenders page](https://github.com/apache/logging-log4j2/blob/master/src/site/xdoc/manual/appenders.xml). The basic format is HTML currently. You can just follow the patterns established by all the other appenders in that page.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
Note that the idea with the "5" naming is to have both modules log4j-lucene5 now and later log4j-lucene6
Gary
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #82: Add a new LuceneAppender which writes logging even...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj commented on the issue:
https://github.com/apache/logging-log4j2/pull/82
committing each time is very affecting performance.
The practice now is to commit once per minute, commit too often, will lead to performance degradation, the submission interval is too long, will lead to the query can not be queried to the nearest log.
This time interval is dynamically adjustable, we don't think it's very meaningful at the moment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 pull request #82: Add a new LuceneAppender which writes loggi...
Posted by liyuj <gi...@git.apache.org>.
Github user liyuj closed the pull request at:
https://github.com/apache/logging-log4j2/pull/82
---