You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "dingjin (Created) (JIRA)" <ji...@apache.org> on 2012/02/29 08:40:03 UTC
[jira] [Created] (LUCENE-3834) The tokenstream create by
SmartChineseAnalyzer can't reset
The tokenstream create by SmartChineseAnalyzer can't reset
----------------------------------------------------------
Key: LUCENE-3834
URL: https://issues.apache.org/jira/browse/LUCENE-3834
Project: Lucene - Java
Issue Type: Bug
Components: modules/analysis
Affects Versions: 3.5
Reporter: dingjin
That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
oringal code
public final class SentenceTokenizer extends Tokenizer {
....
@Override
public void reset() throws IOException {
super.reset();
tokenStart = tokenEnd = 0;
}
...
}
this method should changes as follow
public void reset() throws IOException {
super.reset();
//should reset input
input.reset();
tokenStart = tokenEnd = 0;
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3834) The tokenstream create by
SmartChineseAnalyzer can't reset
Posted by "dingjin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dingjin updated LUCENE-3834:
----------------------------
Priority: Minor (was: Major)
Description:
That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
oringal code
public final class SentenceTokenizer extends Tokenizer {
....
@Override
public void reset() throws IOException {
super.reset();
tokenStart = tokenEnd = 0;
}
...
}
this method should changes as follow
public void reset() throws IOException {
super.reset();
/*should reset input*/
if (input.markSupported())
input.reset();
tokenStart = tokenEnd = 0;
}
was:
That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
oringal code
public final class SentenceTokenizer extends Tokenizer {
....
@Override
public void reset() throws IOException {
super.reset();
tokenStart = tokenEnd = 0;
}
...
}
this method should changes as follow
public void reset() throws IOException {
super.reset();
/*should reset input*/
input.reset();
tokenStart = tokenEnd = 0;
}
> The tokenstream create by SmartChineseAnalyzer can't reset
> ----------------------------------------------------------
>
> Key: LUCENE-3834
> URL: https://issues.apache.org/jira/browse/LUCENE-3834
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Affects Versions: 3.5
> Reporter: dingjin
> Priority: Minor
>
> That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
> They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
> class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
> oringal code
> public final class SentenceTokenizer extends Tokenizer {
> ....
> @Override
> public void reset() throws IOException {
> super.reset();
> tokenStart = tokenEnd = 0;
> }
> ...
> }
> this method should changes as follow
>
> public void reset() throws IOException {
> super.reset();
> /*should reset input*/
> if (input.markSupported())
> input.reset();
> tokenStart = tokenEnd = 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3834) The tokenstream create by
SmartChineseAnalyzer can't reset
Posted by "dingjin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dingjin updated LUCENE-3834:
----------------------------
Description:
That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
oringal code
public final class SentenceTokenizer extends Tokenizer {
....
@Override
public void reset() throws IOException {
super.reset();
tokenStart = tokenEnd = 0;
}
...
}
this method should changes as follow
public void reset() throws IOException {
super.reset();
/*should reset input*/
input.reset();
tokenStart = tokenEnd = 0;
}
was:
That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
oringal code
public final class SentenceTokenizer extends Tokenizer {
....
@Override
public void reset() throws IOException {
super.reset();
tokenStart = tokenEnd = 0;
}
...
}
this method should changes as follow
public void reset() throws IOException {
super.reset();
//should reset input
input.reset();
tokenStart = tokenEnd = 0;
}
> The tokenstream create by SmartChineseAnalyzer can't reset
> ----------------------------------------------------------
>
> Key: LUCENE-3834
> URL: https://issues.apache.org/jira/browse/LUCENE-3834
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Affects Versions: 3.5
> Reporter: dingjin
>
> That is because the field input in class SentenceTokenizer isn't reset after we call the method reset().
> They are two input field,one is from Tokenizer and another is from TokenFilter,if we need to reset a tokenstream created by SmartChineseAnalyzer, both them need reset.This bug is because of the author forget reset input field in class SentenceTokenizer .
> class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer
> oringal code
> public final class SentenceTokenizer extends Tokenizer {
> ....
> @Override
> public void reset() throws IOException {
> super.reset();
> tokenStart = tokenEnd = 0;
> }
> ...
> }
> this method should changes as follow
>
> public void reset() throws IOException {
> super.reset();
> /*should reset input*/
> input.reset();
> tokenStart = tokenEnd = 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org