You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Michael Wechner <mi...@wyona.com> on 2011/12/15 08:59:39 UTC
How to delete individual entries from a log file
Hi
As some of you might now Apache Lucene creates a "write.lock" when an
IndexWriter is updating the index,
hence it can happen that another IndexWriter (inside another thread)
won't be able to update the same index at the same time and based on a
specific timeout something might not get indexed.
We are currently logging all such instances when something does not get
indexed.
Now based on that log file, containing hints re files which have not
been indexed yet, we would like to introduce
a process starting to re-index these specific files and if successful,
then we would like to remove the corresponding particular entry from the
log file.
I understand that log4j might not be intended for such a usecase and
maybe we shouldn't try to misuse it, but I would appreciate any pointers
or ideas or alternatives.
Thanks
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: How to delete individual entries from a log file
Posted by Michael Erskine <ms...@googlemail.com>.
Rather than your log file processor removing items from a log file,
have _it_ log the items that it _has_ re-indexed!
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: How to delete individual entries from a log file
Posted by Michael Wechner <mi...@wyona.com>.
Am 15.12.11 09:32, schrieb Tim Watts:
> I'll try to refrain from chastising you for using a log as a work
> queue... :-)
thank you :-)
>
> but I suppose the "cleanest" way would be to log to a database and then
> delete those rows once the files have been indexed.
I understand, but as you mention below I would like to have as little
moving parts as possible
and log4j is just so nicely simple, but maybe it is too simple for such
a case.
Thanks very much for your feedback
Michael
>
> Alternatively, if that adds too many moving parts and you want to stick
> with text files, leave a space in the log records and rewrite them with
> a marker (e.g. "*") in that space once the files have be indexed to mark
> them as "deleted" or "handled" as the case may be.
>
>
> On Thu, 2011-12-15 at 08:59 +0100, Michael Wechner wrote:
>> Hi
>>
>> As some of you might now Apache Lucene creates a "write.lock" when an
>> IndexWriter is updating the index,
>> hence it can happen that another IndexWriter (inside another thread)
>> won't be able to update the same index at the same time and based on a
>> specific timeout something might not get indexed.
>>
>> We are currently logging all such instances when something does not get
>> indexed.
>>
>> Now based on that log file, containing hints re files which have not
>> been indexed yet, we would like to introduce
>> a process starting to re-index these specific files and if successful,
>> then we would like to remove the corresponding particular entry from the
>> log file.
>>
>> I understand that log4j might not be intended for such a usecase and
>> maybe we shouldn't try to misuse it, but I would appreciate any pointers
>> or ideas or alternatives.
>>
>> Thanks
>>
>> Michael
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: How to delete individual entries from a log file
Posted by Tim Watts <ti...@cliftonfarm.org>.
I'll try to refrain from chastising you for using a log as a work
queue... :-)
but I suppose the "cleanest" way would be to log to a database and then
delete those rows once the files have been indexed.
Alternatively, if that adds too many moving parts and you want to stick
with text files, leave a space in the log records and rewrite them with
a marker (e.g. "*") in that space once the files have be indexed to mark
them as "deleted" or "handled" as the case may be.
On Thu, 2011-12-15 at 08:59 +0100, Michael Wechner wrote:
> Hi
>
> As some of you might now Apache Lucene creates a "write.lock" when an
> IndexWriter is updating the index,
> hence it can happen that another IndexWriter (inside another thread)
> won't be able to update the same index at the same time and based on a
> specific timeout something might not get indexed.
>
> We are currently logging all such instances when something does not get
> indexed.
>
> Now based on that log file, containing hints re files which have not
> been indexed yet, we would like to introduce
> a process starting to re-index these specific files and if successful,
> then we would like to remove the corresponding particular entry from the
> log file.
>
> I understand that log4j might not be intended for such a usecase and
> maybe we shouldn't try to misuse it, but I would appreciate any pointers
> or ideas or alternatives.
>
> Thanks
>
> Michael
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>