You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/05/19 02:18:37 UTC
[GitHub] [lucene] mocobeta opened a new pull request, #904: LUCENE-10312: Revert changes in PersianAnalyzer
mocobeta opened a new pull request, #904:
URL: https://github.com/apache/lucene/pull/904
This reverts changes in PersianAnalyzer #540 from 9x branch.
Users who want to use the new PersianStemmer in 9.x will be able to customer analyzer on their own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta commented on a diff in pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta commented on code in PR #904:
URL: https://github.com/apache/lucene/pull/904#discussion_r876545144
##########
lucene/analysis/common/src/test/org/apache/lucene/analysis/fa/TestPersianStemFilter.java:
##########
@@ -32,7 +32,14 @@ public class TestPersianStemFilter extends BaseTokenStreamTestCase {
@Override
public void setUp() throws Exception {
super.setUp();
- a = new PersianAnalyzer();
+ a =
+ new Analyzer() {
+ @Override
+ protected TokenStreamComponents createComponents(String fieldName) {
+ final Tokenizer source = new MockTokenizer();
+ return new TokenStreamComponents(source, new PersianStemFilter(source));
+ }
+ };
Review Comment:
This is needed to make TestPersianStemFilter work, it could be better to forward port to main so that the test does not depend on PersianAnalyzer implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta commented on a diff in pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta commented on code in PR #904:
URL: https://github.com/apache/lucene/pull/904#discussion_r876540849
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/fa/PersianAnalyzer.java:
##########
@@ -136,11 +121,7 @@ protected TokenStreamComponents createComponents(String fieldName) {
* the order here is important: the stopword list is normalized with the
* above!
*/
- result = new StopFilter(result, stopwords);
- if (!stemExclusionSet.isEmpty()) {
- result = new SetKeywordMarkerFilter(result, stemExclusionSet);
- }
- return new TokenStreamComponents(source, new PersianStemFilter(result));
+ return new TokenStreamComponents(source, new StopFilter(result, stopwords));
Review Comment:
Returned TokenStreamComponents is the same as in 9.1.
https://github.com/apache/lucene/blob/1bf3cbc0b9d11a35bf8b655f9cb5ff6c11889dbf/lucene/analysis/common/src/java/org/apache/lucene/analysis/fa/PersianAnalyzer.java#L124
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta commented on pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta commented on PR #904:
URL: https://github.com/apache/lucene/pull/904#issuecomment-1131633700
Thank you both!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta commented on a diff in pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta commented on code in PR #904:
URL: https://github.com/apache/lucene/pull/904#discussion_r877002862
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/fa/PersianAnalyzer.java:
##########
@@ -136,11 +121,7 @@ protected TokenStreamComponents createComponents(String fieldName) {
* the order here is important: the stopword list is normalized with the
* above!
*/
- result = new StopFilter(result, stopwords);
- if (!stemExclusionSet.isEmpty()) {
- result = new SetKeywordMarkerFilter(result, stemExclusionSet);
- }
- return new TokenStreamComponents(source, new PersianStemFilter(result));
+ return new TokenStreamComponents(source, new StopFilter(result, stopwords));
Review Comment:
OK, I'll update the javadocs in main later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta merged pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta merged PR #904:
URL: https://github.com/apache/lucene/pull/904
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on a diff in pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
rmuir commented on code in PR #904:
URL: https://github.com/apache/lucene/pull/904#discussion_r877000127
##########
lucene/analysis/common/src/java/org/apache/lucene/analysis/fa/PersianAnalyzer.java:
##########
@@ -136,11 +121,7 @@ protected TokenStreamComponents createComponents(String fieldName) {
* the order here is important: the stopword list is normalized with the
* above!
*/
- result = new StopFilter(result, stopwords);
- if (!stemExclusionSet.isEmpty()) {
- result = new SetKeywordMarkerFilter(result, stemExclusionSet);
- }
- return new TokenStreamComponents(source, new PersianStemFilter(result));
+ return new TokenStreamComponents(source, new StopFilter(result, stopwords));
Review Comment:
I guess no javadocs changes were needed for this method because they never got updated in the first place :) Maybe something to fix in 10.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mocobeta commented on pull request #904: LUCENE-10312: Revert changes in PersianAnalyzer
Posted by GitBox <gi...@apache.org>.
mocobeta commented on PR #904:
URL: https://github.com/apache/lucene/pull/904#issuecomment-1131635115
I'll cherry-pick this in branch_9_2 soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org