You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael Busch (JIRA)" <ji...@apache.org> on 2007/05/15 13:24:16 UTC
[jira] Created: (LUCENE-880) DocumentWriter closes TokenStreams too
early
DocumentWriter closes TokenStreams too early
--------------------------------------------
Key: LUCENE-880
URL: https://issues.apache.org/jira/browse/LUCENE-880
Project: Lucene - Java
Issue Type: Bug
Reporter: Michael Busch
Assigned To: Michael Busch
Priority: Minor
The DocumentWriter closes a TokenStream as soon as it has consumed its tokens. The javadoc of TokenStream.close() says that it releases resources associated with the stream. However, the DocumentWriter keeps references of the resources (i. e. payload byte arrays, term strings) until it writes the postings to the new segment, which means that DocumentWriter should call TokenStream.close() after it has written the postings.
This problem occurs in multithreaded applications where e. g. pooling is used for the resources. My patch adds a new test to TestPayloads which shows this problem. Multiple threads add documents with payloads to an index and use a pool of byte arrays for the payloads. TokenStream.close() puts the byte arrays back into the pool. The test fails with the old version but runs successfully with the patched version.
All other units tests pass as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-880) DocumentWriter closes TokenStreams
too early
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Busch resolved LUCENE-880.
----------------------------------
Resolution: Fixed
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Committed.
> DocumentWriter closes TokenStreams too early
> --------------------------------------------
>
> Key: LUCENE-880
> URL: https://issues.apache.org/jira/browse/LUCENE-880
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael Busch
> Assigned To: Michael Busch
> Priority: Minor
> Attachments: lucene-880.patch
>
>
> The DocumentWriter closes a TokenStream as soon as it has consumed its tokens. The javadoc of TokenStream.close() says that it releases resources associated with the stream. However, the DocumentWriter keeps references of the resources (i. e. payload byte arrays, term strings) until it writes the postings to the new segment, which means that DocumentWriter should call TokenStream.close() after it has written the postings.
> This problem occurs in multithreaded applications where e. g. pooling is used for the resources. My patch adds a new test to TestPayloads which shows this problem. Multiple threads add documents with payloads to an index and use a pool of byte arrays for the payloads. TokenStream.close() puts the byte arrays back into the pool. The test fails with the old version but runs successfully with the patched version.
> All other units tests pass as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-880) DocumentWriter closes TokenStreams too
early
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Busch updated LUCENE-880:
---------------------------------
Attachment: lucene-880.patch
Patch with new unit test.
> DocumentWriter closes TokenStreams too early
> --------------------------------------------
>
> Key: LUCENE-880
> URL: https://issues.apache.org/jira/browse/LUCENE-880
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael Busch
> Assigned To: Michael Busch
> Priority: Minor
> Attachments: lucene-880.patch
>
>
> The DocumentWriter closes a TokenStream as soon as it has consumed its tokens. The javadoc of TokenStream.close() says that it releases resources associated with the stream. However, the DocumentWriter keeps references of the resources (i. e. payload byte arrays, term strings) until it writes the postings to the new segment, which means that DocumentWriter should call TokenStream.close() after it has written the postings.
> This problem occurs in multithreaded applications where e. g. pooling is used for the resources. My patch adds a new test to TestPayloads which shows this problem. Multiple threads add documents with payloads to an index and use a pool of byte arrays for the payloads. TokenStream.close() puts the byte arrays back into the pool. The test fails with the old version but runs successfully with the patched version.
> All other units tests pass as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org