You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Karl Wettin (JIRA)" <ji...@apache.org> on 2008/01/25 14:44:39 UTC
[jira] Created: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
BoostingTermQuery#defaultTermBoost
----------------------------------
Key: LUCENE-1155
URL: https://issues.apache.org/jira/browse/LUCENE-1155
Project: Lucene - Java
Issue Type: Improvement
Components: Search
Reporter: Karl Wettin
Priority: Trivial
This patch allows a null payload to mean something different than 1f.
(I have this use case where 99% of my tokens share the same rather large token position payload boost.)
{code}
+++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
@@ -41,11 +41,16 @@
*/
public class BoostingTermQuery extends SpanTermQuery{
+ private Float defaultTermBoost = null;
public BoostingTermQuery(Term term) {
super(term);
}
+ public BoostingTermQuery(Term term, Float defaultTermBoost) {
+ super(term);
+ this.defaultTermBoost = defaultTermBoost;
+ }
protected Weight createWeight(Searcher searcher) throws IOException {
return new BoostingTermWeight(this, searcher);
@@ -107,7 +112,9 @@
payload = positions.getPayload(payload, 0);
payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
payloadsSeen++;
-
+ } else if (defaultTermBoost != null) {
+ payloadScore += defaultTermBoost;
+ payloadsSeen++;
} else {
//zero out the payload?
}
@@ -146,7 +153,14 @@
}
+ public Float getDefaultTermBoost() {
+ return defaultTermBoost;
+ }
+ public void setDefaultTermBoost(Float defaultTermBoost) {
+ this.defaultTermBoost = defaultTermBoost;
+ }
+
public boolean equals(Object o) {
if (!(o instanceof BoostingTermQuery))
return false;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630687#action_12630687 ]
Karl Wettin commented on LUCENE-1155:
-------------------------------------
bq. Sorry, I still don't get it, but I'm probably dense. Maybe an actual example would help.
In my single case all term positions contains a payload boost. In order to optimize the index size I analyzed the payloads and came to the conclution that 30% or so had the boost 5f, a clear majority of the boosts. In order to minimize the index size I choosed to use this patch and set the default payload to 5f rather than storing 5f in the payload.
How and why I managed to get a majority of the payloads to be 5f and if I could avoid this by manageing boosts in another layer is a completely different story.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630761#action_12630761 ]
Grant Ingersoll commented on LUCENE-1155:
-----------------------------------------
How is that any different than marking the 70% as having a boost ranging between 1/5 = 0.2 and 5/5 = 1 whereby no patch is needed? You still have the same 30% null payloads
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625160#action_12625160 ]
Karl Wettin commented on LUCENE-1155:
-------------------------------------
I still think this patch makes sense and would very much like to see it in 2.4.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630765#action_12630765 ]
Karl Wettin commented on LUCENE-1155:
-------------------------------------
bq. How is that any different than marking the 70% as having a boost ranging between 1/5 = 0.2 and 5/5 = 1 whereby no patch is needed? You still have the same 30% null payloads
There would be absolutly no difference to the end result.
I want to turn this question around and ask why you don't normalize all boosts at query time to top index 1f and then use negative boost for everything else?
It's about gutfeelings.
Nulling out the payloads could be gained by dividing the weight by 5 at creation time (due to lots of reasons this was not a good chooise), by a dividing all weight by 5 in a filter in the end of the chain, or by introducing this patch. For me BoostingTermQuery felt like the most obvious place to handle this stuff, and it still does.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562497#action_12562497 ]
Grant Ingersoll commented on LUCENE-1155:
-----------------------------------------
I think I am missing something here. If they are all the same, except a few, why even store a payload for those that are the same? Why not just rely on the usual term scoring for those and just get the boost from those tokens that truly need a payload?
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562514#action_12562514 ]
Karl Wettin commented on LUCENE-1155:
-------------------------------------
{quote}
I think I am missing something here. If they are all the same, except a few, why even store a payload for those that are the same? Why not just rely on the usual term scoring for those and just get the boost from those tokens that truly need a payload?
{quote}
Thanks to this patch I don't store payload for all those that are the same.
I only store the payload for 1% of the tokens. The remaining 99% of the term positions, them with a null payload, are boosted according to a value I configure my query to use. In my case I want a null payload to mean 5f boost, and the 1% positions with payload set have something between 4.9f and 1f boost.
I'm not at all surprised if it is considered backwards, works great though. I have no idea to how one can do this another way.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630638#action_12630638 ]
Michael McCandless commented on LUCENE-1155:
--------------------------------------------
Grant, what do you think of this one? The patch looks reasonable to me; it's backwards compatible. Should we do this for 2.4?
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1155:
--------------------------------
Description:
This patch allows a null payload to mean something different than 1f.
(I have this use case where 99% of my tokens share the same rather large token position payload boost.)
{code}
Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
===================================================================
--- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
+++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
@@ -41,11 +41,16 @@
*/
public class BoostingTermQuery extends SpanTermQuery{
+ private Float defaultTermBoost = null;
public BoostingTermQuery(Term term) {
super(term);
}
+ public BoostingTermQuery(Term term, Float defaultTermBoost) {
+ super(term);
+ this.defaultTermBoost = defaultTermBoost;
+ }
protected Weight createWeight(Searcher searcher) throws IOException {
return new BoostingTermWeight(this, searcher);
@@ -107,7 +112,9 @@
payload = positions.getPayload(payload, 0);
payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
payloadsSeen++;
-
+ } else if (defaultTermBoost != null) {
+ payloadScore += defaultTermBoost;
+ payloadsSeen++;
} else {
//zero out the payload?
}
@@ -146,7 +153,14 @@
}
+ public Float getDefaultTermBoost() {
+ return defaultTermBoost;
+ }
+ public void setDefaultTermBoost(Float defaultTermBoost) {
+ this.defaultTermBoost = defaultTermBoost;
+ }
+
public boolean equals(Object o) {
if (!(o instanceof BoostingTermQuery))
return false;
{code}
was:
This patch allows a null payload to mean something different than 1f.
(I have this use case where 99% of my tokens share the same rather large token position payload boost.)
{code}
+++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
@@ -41,11 +41,16 @@
*/
public class BoostingTermQuery extends SpanTermQuery{
+ private Float defaultTermBoost = null;
public BoostingTermQuery(Term term) {
super(term);
}
+ public BoostingTermQuery(Term term, Float defaultTermBoost) {
+ super(term);
+ this.defaultTermBoost = defaultTermBoost;
+ }
protected Weight createWeight(Searcher searcher) throws IOException {
return new BoostingTermWeight(this, searcher);
@@ -107,7 +112,9 @@
payload = positions.getPayload(payload, 0);
payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
payloadsSeen++;
-
+ } else if (defaultTermBoost != null) {
+ payloadScore += defaultTermBoost;
+ payloadsSeen++;
} else {
//zero out the payload?
}
@@ -146,7 +153,14 @@
}
+ public Float getDefaultTermBoost() {
+ return defaultTermBoost;
+ }
+ public void setDefaultTermBoost(Float defaultTermBoost) {
+ this.defaultTermBoost = defaultTermBoost;
+ }
+
public boolean equals(Object o) {
if (!(o instanceof BoostingTermQuery))
return false;
{code}
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
missed first two lines in the diff
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630655#action_12630655 ]
Grant Ingersoll commented on LUCENE-1155:
-----------------------------------------
Sorry, I still don't get it, but I'm probably dense. Maybe an actual example would help.
Next, it incs payloadsSeen when it hasn't, in fact, seen a payload. That seems weird.
Second, why not just scale the results from 1 to 5 to be between 0 and 1 (and possibly just call Query.setBoost(5))? That is, for the 1% of positions, make it range between 0.9 and 0.1 (or whatever).
That being said, I suppose it doesn't hurt, but it just seems confusing. If anything, I would rename the variable something like nullPayloadBoost or something else, as I think defaultTermBoost implies it's the default boost applied when there is a payload, but the boost isn't specified, which doesn't make sense either.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1155) BoostingTermQuery#defaultTermBoost
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571789#action_12571789 ]
Karl Wettin commented on LUCENE-1155:
-------------------------------------
{quote}
I think I am missing something here. If they are all the same, except a few, why even store a payload for those that are the same? Why not just rely on the usual term scoring for those and just get the boost from those tokens that truly need a payload?
{quote}
It might be worth mentioning that I have omitted the norms.
> BoostingTermQuery#defaultTermBoost
> ----------------------------------
>
> Key: LUCENE-1155
> URL: https://issues.apache.org/jira/browse/LUCENE-1155
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Karl Wettin
> Priority: Trivial
>
> This patch allows a null payload to mean something different than 1f.
> (I have this use case where 99% of my tokens share the same rather large token position payload boost.)
> {code}
> Index: src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java
> ===================================================================
> --- src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (revision 615215)
> +++ src/java/org/apache/lucene/search/payloads/BoostingTermQuery.java (working copy)
> @@ -41,11 +41,16 @@
> */
> public class BoostingTermQuery extends SpanTermQuery{
>
> + private Float defaultTermBoost = null;
>
> public BoostingTermQuery(Term term) {
> super(term);
> }
>
> + public BoostingTermQuery(Term term, Float defaultTermBoost) {
> + super(term);
> + this.defaultTermBoost = defaultTermBoost;
> + }
>
> protected Weight createWeight(Searcher searcher) throws IOException {
> return new BoostingTermWeight(this, searcher);
> @@ -107,7 +112,9 @@
> payload = positions.getPayload(payload, 0);
> payloadScore += similarity.scorePayload(term.field(), payload, 0, positions.getPayloadLength());
> payloadsSeen++;
> -
> + } else if (defaultTermBoost != null) {
> + payloadScore += defaultTermBoost;
> + payloadsSeen++;
> } else {
> //zero out the payload?
> }
> @@ -146,7 +153,14 @@
>
> }
>
> + public Float getDefaultTermBoost() {
> + return defaultTermBoost;
> + }
>
> + public void setDefaultTermBoost(Float defaultTermBoost) {
> + this.defaultTermBoost = defaultTermBoost;
> + }
> +
> public boolean equals(Object o) {
> if (!(o instanceof BoostingTermQuery))
> return false;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org