You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/12/15 10:37:29 UTC
[GitHub] [lucene] nosvalds opened a new issue, #12021: Large fields with large="true" can be truncated in v9+
nosvalds opened a new issue, #12021:
URL: https://github.com/apache/lucene/issues/12021
### Description
## Issue
For fields using `large="true"`, large fields (which is what they are intended for) can be truncated in v9+ of Lucene.
Example fieldtype definition:
```
<fieldtype name="string_large" class="solr.TextField" multiValued="false" indexed="false" stored="true" omitNorms="true" large="true" />
```
## Cause
Looks like this is a bug introduced along with [LUCENE-8805](https://issues.apache.org/jira/browse/LUCENE-8805) / https://github.com/apache/lucene/issues/9849:
[https://github.com/apache/lucene/blob/5a694ea26ff862ecc874ca798135073d300c2234/sol[…]r/core/src/java/org/apache/solr/search/SolrDocumentFetcher.java](https://github.com/apache/lucene/blob/5a694ea26ff862ecc874ca798135073d300c2234/solr/core/src/java/org/apache/solr/search/SolrDocumentFetcher.java#L462-L465)
Specifically with respect to "large" fields handling.
The length in utf8 bytes will often be longer than the string length `value.length()`, hence the truncation.
## Fix
The Fix would be:
`bytesRef.length = bytesRef.bytes.length`
### Version and environment details
- Solr v9.1.0
- Lucene v9.3.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] nosvalds closed issue #12021: Large fields with large="true" can be truncated in v9+
Posted by GitBox <gi...@apache.org>.
nosvalds closed issue #12021: Large fields with large="true" can be truncated in v9+
URL: https://github.com/apache/lucene/issues/12021
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #12021: Large fields with large="true" can be truncated in v9+
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #12021:
URL: https://github.com/apache/lucene/issues/12021#issuecomment-1352977757
This looks like a bug in solr code (SolrDocumentFetcher) so I'd recommend opening a bug over at https://github.com/apache/solr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] nosvalds commented on issue #12021: Large fields with large="true" can be truncated in v9+
Posted by GitBox <gi...@apache.org>.
nosvalds commented on issue #12021:
URL: https://github.com/apache/lucene/issues/12021#issuecomment-1353077520
Sorry about that looks like the code link I had was from before the split. Moved this issue here: https://issues.apache.org/jira/browse/SOLR-16589
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org