[jira] [Comment Edited] (SOLR-5330) PerSegmentSingleValuedFaceting overwrites facet values

["Yonik Seeley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793213#comment-13793213 ] 

Yonik Seeley edited comment on SOLR-5330 at 10/12/13 2:30 AM:
--------------------------------------------------------------

So I instrumented the faceting code like so:
{code}
          seg.tempBR = seg.tenum.next();
if (seg.tempBR.bytes == val.bytes) {
System.err.println("##########SHARING DETECTED: val.offset="+val.offset + " val.length="+val.length + " new.offset="+seg.tempBR.offset + " new.length="+seg.tempBR.length);
if (val.offset == seg.tempBR.offset) {
  System.err.println("!!!!!!SHARING USING SAME OFFSET");
}
{code}

And it detects tons of sharing (the returned bytesref still pointing to the same byte[]) of course... but the thing is, it never generates an invalid result.  calling next() on the term enum never changes the bytes that were previously pointed to... it simply points to a different part of the same byte array.  I can never detect a case where the original bytes are changed, thus invalidating the shallow copy.

Example output:
{code}
##########SHARING DETECTED: val.offset=1 val.length=4 new.offset=6 new.length=4
{code}


was (Author: yseeley@gmail.com):
So I instrumented the faceting code like so:
{code}
          seg.tempBR = seg.tenum.next();
if (seg.tempBR.bytes == val.bytes) {
System.err.println("##########SHARING DETECTED: val.offset="+val.offset + " val.length="+val.length + " new.offset="+seg.tempBR.offset + " new.length="+seg.tempBR.length);
if (val.offset == seg.tempBR.offset) {
  System.err.println("!!!!!!SHARING USING SAME OFFSET");
}
{code}

And it detects tons of sharing (the returned bytesref still pointing to the same byte[]) of course... but the thing is, it never generates an invalid result.  calling next() on the term enum never changes the bytes that were previously pointed to... it simply points to a different part of the same byte array.  I can never detect a case where the original bytes are changed, thus invalidating the shallow copy.


> PerSegmentSingleValuedFaceting overwrites facet values
> ------------------------------------------------------
>
>                 Key: SOLR-5330
>                 URL: https://issues.apache.org/jira/browse/SOLR-5330
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.2.1
>            Reporter: Michael Froh
>            Assignee: Yonik Seeley
>         Attachments: solr-5330.patch
>
>
> I recently tried enabling facet.method=fcs for one of my indexes and found a significant performance improvement (with a large index, many facet values, and near-realtime updates). Unfortunately, the results were also wrong. Specifically, some facet values were being partially overwritten by other facet values. (That is, if I expected facet values like "abcdef" and "123", I would get a value like "123def".)
> Debugging through the code, it looks like the problem was in PerSegmentSingleValuedFaceting, specifically in the getFacetCounts method, when BytesRef val is shallow-copied from the temporary per-segment BytesRef. The byte array assigned to val is shared with the byte array for seg.tempBR, and is overwritten a few lines down by the call to seg.tenum.next().
> I managed to fix it locally by replacing the shallow copy with a deep copy.
> While I encountered this problem on Solr 4.2.1, I see that the code is identical in 4.5. Unless the behavior of TermsEnum.next() has changed, I believe this bug still exists.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org