You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by smartprix <gi...@git.apache.org> on 2015/11/18 19:57:41 UTC
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
GitHub user smartprix opened a pull request:
https://github.com/apache/lucene-solr/pull/210
WordDelimiterFilter - Don't split words marked as keyword
Currently WordDelimiterFilter also splits keywords into tokens. eg. if 128GB is maked as a keyword using KeywordMarkerFilter, WordDelimiterFilter would still split it into 128 and GB, while ideally it should not split as it is a keyword.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/smartprix/lucene-solr trunk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucene-solr/pull/210.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #210
----
commit 50385c79e13c32226e92234b411cf0f2a80cae1b
Author: smartprix <ad...@smartprix.com>
Date: 2015-11-18T18:45:37Z
WordDelimiterFilter - Don't split word marked as keyword
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by elyograg <gi...@git.apache.org>.
Github user elyograg commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-157833770
I like this idea. I think I'd go one step further -- make this behavior configurable with an attribute named something like skipKeywordTokens. It should default to true when luceneMatchVersion is 6.0 or higher, and false for anything lower.
FYI, the lucene-solr github repository is a read-only mirror of the project in Apache's subversion repository. Apache's Jira installation is the official bugtracker.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by elyograg <gi...@git.apache.org>.
Github user elyograg commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158099893
I think you've got the default values backwards, but other than that it looks good to me. For 5.x, we need the default to be false so existing setups do not change behavior, but it is sensible to change the default to true with 6.0.
We can use github patches, but cannot use the pull request directly or do the commit on github. If you can create an issue in Jira, that will be helpful.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by elyograg <gi...@git.apache.org>.
Github user elyograg commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158115946
Now that I've gotten a closer look, I see that you went with splitKeywordTokens rather than skipKeywordTokens, so your defaults were correct.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by LucVL <gi...@git.apache.org>.
Github user LucVL commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158335822
There’s good documentation on combining git pull-requests with jira issues on the apache wiki (there’s a bot for that)
https://wiki.apache.org/lucene-java/BensonMarguliesGitWorkflow
Basically:
- Create the JIRA issue first
- Create the pull request with the JIRA issue in the title. This will cause a bot to pick it up and link the two. A comment will appear in the JIRA issue that includes a link to an equivalent patch for non-git users.
- When the JIRA issue is closed, the committer should include “This closes #PP”. This causes the bot to close the PR as well
For an example, see https://issues.apache.org/jira/browse/SOLR-8050
Luc
From: smartprix [mailto:notifications@github.com]
Sent: donderdag 19 november 2015 15:27
To: apache/lucene-solr
Subject: Re: [lucene-solr] WordDelimiterFilter - Don't split words marked as keyword (#210)
The behavior should is now configurable. I have updated the pull request to reflect that. A new attribute "splitKeywordTokens" which is false by default for lucene >= 6.0 and true otherwise.
Does lucene not accept pull requests from github? Should I create it on JIRA?
—
Reply to this email directly or view it on GitHub<https://github.com/apache/lucene-solr/pull/210#issuecomment-158072439>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by smartprix <gi...@git.apache.org>.
Github user smartprix commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158072439
The behavior should is now configurable. I have updated the pull request to reflect that. A new attribute "splitKeywordTokens" which is false by default for lucene >= 6.0 and true otherwise.
Does lucene not accept pull requests from github? Should I create it on JIRA?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by elyograg <gi...@git.apache.org>.
Github user elyograg commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158147256
I've created the issue in Jira. Let's continue the discussion there. When the patch is ready, I will commit it with the keywords that will close this PR.
https://issues.apache.org/jira/browse/LUCENE-6903
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request: WordDelimiterFilter - Don't split words ...
Posted by elyograg <gi...@git.apache.org>.
Github user elyograg commented on the pull request:
https://github.com/apache/lucene-solr/pull/210#issuecomment-158117696
What name would you like to have in the CHANGES.txt giving you credit for the change?
I believe the only thing that's missing from the patch now is something in TestWordDelimiter and maybe TestWordDelimiterFactory that makes sure the new functionality is working as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org