You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Kamal Kishore Aggarwal <kk...@gmail.com> on 2024/06/22 09:07:57 UTC
Highlight snippets slightly varying in solr upgrade from 8.8.2 to 9.1.1
Hi All,
Currently, we are using solr 8.8.2 version and have implemented
highlighting feature on 2 fields - KIN (title) and SNPT (description) with
the *fastvector* method.
Below are the highlight params:
[image: image.png]
While upgrading solr from 8.8.2 to 9.1.1 version, we are observing few
minor discrepancies in the highlight snippets. FYI, there were certain
documents missing in solr 9.1.1, the data is not the same in both clusters.
Now, when I tested manually for certain queries to fetch 10 documents from
both the solr cluster and compare the snippets from both, here are the
observations:
1. The result count is the same for these queries, I have excluded the
missing data using the filter query.
2. The highlight snippet of KIN field which is title like field is the
same i.e the highlights matched for 45-50 documents which I checked
manually.
*3. One of the highlighted snippets out of 2 snippets of theSNPT field,
which is a description-like field, varies for 5-10% of documents which I
checked manually. (This is an issue).*
Furthermore,
1. Took a document where highlights for Snippets in AWS and On-prem
were different and indexed that document.
2. Lets say *solr8v1* is the collection with only one record on solr
8.8.2, *solr9v1* is the collection with only one record on solr 9.1.1
version. Lets day *solr8vN* is the collection with all records on solr
8.8.2, *solr9v1* is the collection with all records on solr 9.1.1.
3. Indexed that separated document in both the new collections -
*solr8v1* and *solr9v1* .
4. Ran queries on both 4 collections.
Observations
HL of the same document are different on *solr8v1* * and * *solr8vN**.
Infact, HL snippets of 4 collections for the same query **are different**.*
Below are the different highlights for same query on different collections:
1. solr8v1 - Collection with only 1 document on Solr 8.8.2
"highlights" : {
"SNPT" : [ {
"value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
1:NA <br"
}, {
"value" : "br />Below error is coming in prod when cust
<font>microservice</font> is calling party-organization MS and in case"
} ]
}
2. solr8vN - Collection with all documents on Solr 8.8.2
"highlights" : {
"SNPT" : [ {
"value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
1:NA <br"
}, {
"value" : " As per PMX 22.07 account should be using SDK
<font>Chassis</font> 7.1.z 
\nWhy are you using 6.12.z ?
\nhttps"
} ]
}
3. solr9v1 - Collection with only 1 document on Solr 9.1.1
"highlights" : {
"SNPT" : [ {
"value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
1:NA <br"
}, {
"value" : " As per PMX 22.07 account should be using SDK
<font>Chassis</font> 7.1.z 
\nWhy are you using 6.12.z ?
\nhttps"
} ]
}
4. solr9vN - Collection with all documents on Solr 9.1.1 (i.e this
collection has less data than solr 8)
"highlights" : {
"SNPT" : [ {
"value" : "br />Below error is coming in prod when cust
<font>microservice</font> is calling party-organization MS and in case"
}, {
"value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
1:NA <br"
} ]
}
Also, I went through solr release notes also to check if there are any
changes in solr Highlighting algorithm, but could not find anything related
to it in release notes. Please confirm if this minor highlight difference
is expected in solr upgrade and what could be the reason for it.
Regards
Kamal
Re: Highlight snippets slightly varying in solr upgrade from 8.8.2 to 9.1.1
Posted by Kamal Kishore Aggarwal <kk...@gmail.com>.
Hi Solr Experts,
Any comment here.
Regards
Kamal
On Sat, Jun 22, 2024 at 2:37 PM Kamal Kishore Aggarwal <
kkroyal.192@gmail.com> wrote:
> Hi All,
>
> Currently, we are using solr 8.8.2 version and have implemented
> highlighting feature on 2 fields - KIN (title) and SNPT (description) with
> the *fastvector* method.
>
> Below are the highlight params:
>
> [image: image.png]
>
> While upgrading solr from 8.8.2 to 9.1.1 version, we are observing few
> minor discrepancies in the highlight snippets. FYI, there were certain
> documents missing in solr 9.1.1, the data is not the same in both clusters.
> Now, when I tested manually for certain queries to fetch 10 documents from
> both the solr cluster and compare the snippets from both, here are the
> observations:
>
> 1. The result count is the same for these queries, I have excluded the
> missing data using the filter query.
> 2. The highlight snippet of KIN field which is title like field is the
> same i.e the highlights matched for 45-50 documents which I checked
> manually.
> *3. One of the highlighted snippets out of 2 snippets of theSNPT field,
> which is a description-like field, varies for 5-10% of documents which I
> checked manually. (This is an issue).*
>
> Furthermore,
>
> 1. Took a document where highlights for Snippets in AWS and On-prem
> were different and indexed that document.
> 2. Lets say *solr8v1* is the collection with only one record on solr
> 8.8.2, *solr9v1* is the collection with only one record on solr 9.1.1
> version. Lets day *solr8vN* is the collection with all records on
> solr 8.8.2, *solr9v1* is the collection with all records on solr 9.1.1.
> 3. Indexed that separated document in both the new collections -
> *solr8v1* and *solr9v1* .
> 4. Ran queries on both 4 collections.
>
> Observations
> HL of the same document are different on *solr8v1* * and * *solr8vN**.
> Infact, HL snippets of 4 collections for the same query **are different*
> *.*
>
> Below are the different highlights for same query on different collections:
>
> 1. solr8v1 - Collection with only 1 document on Solr 8.8.2
>
> "highlights" : {
> "SNPT" : [ {
> "value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
> version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
> 1:NA <br"
> }, {
> "value" : "br />Below error is coming in prod when cust
> <font>microservice</font> is calling party-organization MS and in case"
> } ]
> }
>
> 2. solr8vN - Collection with all documents on Solr 8.8.2
>
> "highlights" : {
> "SNPT" : [ {
> "value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
> version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
> 1:NA <br"
> }, {
> "value" : " As per PMX 22.07 account should be using SDK
> <font>Chassis</font> 7.1.z 
\nWhy are you using 6.12.z ?
\nhttps"
> } ]
> }
>
> 3. solr9v1 - Collection with only 1 document on Solr 9.1.1
>
> "highlights" : {
> "SNPT" : [ {
> "value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
> version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
> 1:NA <br"
> }, {
> "value" : " As per PMX 22.07 account should be using SDK
> <font>Chassis</font> 7.1.z 
\nWhy are you using 6.12.z ?
\nhttps"
> } ]
> }
>
> 4. solr9vN - Collection with all documents on Solr 9.1.1 (i.e this
> collection has less data than solr 8)
>
> "highlights" : {
> "SNPT" : [ {
> "value" : "br />Below error is coming in prod when cust
> <font>microservice</font> is calling party-organization MS and in case"
> }, {
> "value" : "Consolidate+PMX+22.07 | MS <font>360</font> SDK
> version is 6.12.13 | Please share SDK <font>Chassis</font> version. <p>Step
> 1:NA <br"
> } ]
> }
>
> Also, I went through solr release notes also to check if there are any
> changes in solr Highlighting algorithm, but could not find anything related
> to it in release notes. Please confirm if this minor highlight difference
> is expected in solr upgrade and what could be the reason for it.
>
> Regards
> Kamal
>