You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2019/02/27 20:50:00 UTC
[jira] [Reopened] (SOLR-13261) Make SortableTextField work with
export/streaming
[ https://issues.apache.org/jira/browse/SOLR-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man reopened SOLR-13261:
-----------------------------
re-opening due to concerns about this change...
unlike StrField, SortableTextField does not include any garuntee that it's docValues are usable in results – they may in fact be truncated because of the (default or explicit){{maxCharsForDocValues}}. This is the entire reason why SortableTextField defaults to {{useDocValuesAsStored="false"}} – and is the the reason why a field that explicitly sets {{useDocValuesAsStored="true"}} gets different behavior (and error) at indexing time if {{maxCharsForDocValues}} is exceeded -- see the javadocs (and initial discussion in SOLR-11916) for details.
The _only_ way the DV from SortableTextField should be safe to use in the ExportResponseWriter is if the user explicitly set {{useDocValuesAsStored="true"}} -- because otherwise we have no way of being certain that DV String isn't truncated ... but AFAICT the ExportWriter doesn't do anything to check {{FieldType.useDocValuesAsStored()}} for any field types, let alone SortableTextField, and this change doesn't seem to add that.
If I'm missunderstanding things, or if there is some other place in the code path i'm not realizing that will prevent a {{useDocValuesAsStored="false"}} SortableTextField from being used with ExportWriter then that's fine -- but there should realy be a test _proving_ that attempting this will cause an error.
If i'm correct, then this change should be rolled back unless/until {{useDocValuesAsStored}} is respected by the ExportWriter so that we don't returning missleadingly truncated values
> Make SortableTextField work with export/streaming
> -------------------------------------------------
>
> Key: SOLR-13261
> URL: https://issues.apache.org/jira/browse/SOLR-13261
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 7.7, 8.0, master (9.0)
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
> Fix For: master (9.0), 8.1
>
> Attachments: SOLR-13261.patch, SOLR-13261.patch
>
>
> ExportWriter (and perhaps other places) explicitly tests for certain field types and error out with "Export fields must either be one of the following types: int,float,long,double,string,date,boolean"
> It seems perfectly legal to export SortableTextField as well as it's a DV field. How desirable that would be is an open question.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org