You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Vishal Patel <vi...@outlook.com> on 2024/04/17 12:11:48 UTC
docValues true is not working for solr.TextField in Solr 8.9.0
We are using Solr 8.9.0. We have configured Solr cloud like 2 shards and each shard has one replica. We have used 7 zoo keepers for Solr cloud.
Our schema field is like
<field name="title" type="text_string" indexed="true" stored="true" multiValued="false" omitNorms="true" termVectors="false" termPositions="false" termOffsets="false" omitTermFreqAndPositions="true"/>
<dynamicField name="AIR_ts_s_*" type="text_string" indexed="true" stored="true" omitTermFreqAndPositions="true"/>
<fieldtype name="text_string" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldtype>
We are using sorting and faceting on both field title and AIR_ts_s_100. We see the both field in fieldCache using admin panel and it is taking much memory. After the long time it is not decreasing.
We know the we can not apply docValues true directly in solr.TextField. So, We have taken reference from https://github.com/grossws/solr-dvtf.
It is working fine for title like after applying sort and facet on title field we can not see in fieldCache but when we are applying sort on AIR_ts_s_100 then we are getting ERROR
org.apache.solr.common.SolrException: can not sort on multivalued field: AIR_ts_s_100 of type: text_string
at org.apache.solr.schema.SchemaField.checkSortability(SchemaField.java:188)
at org.apache.solr.schema.FieldType.getSortedSetSortField(FieldType.java:732)
at org.apache.solr.schema.TextField.getSortField(TextField.java:121)
at org.apache.solr.schema.SchemaField.getSortField(SchemaField.java:154)
Can you please suggest us why we are getting ERROR like can not sort on multivalued field? because it is not multi value true.
Our main issue is day by day increasing the fieldCache size. How can we avoid or reduce size for solr.TextField?
Regards,
Vishal
Re: docValues true is not working for solr.TextField in Solr 8.9.0
Posted by Vishal Patel <vi...@outlook.com>.
We resolved the issue after the multiValued="false" because in that jar if multiValued is not assigned then it will take true so we are getting error on sorting time.
@Override
protected void init(IndexSchema schema, Map<String, String> args) {
// fail if docValues==false
if ((falseProperties & DOC_VALUES) != 0) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, getClass().getName() + " requires docValues==true");
}
// force docValues
properties |= DOC_VALUES;
// enable multiValued if not explicitly set to false
if ((falseProperties & MULTIVALUED) == 0) {
properties |= MULTIVALUED;
}
String omitReversed = args.remove(OMIT_WILDCARD_REVERSED_TERMS);
if (omitReversed != null) {
this.omitReversed = Boolean.parseBoolean(omitReversed);
}
super.init(schema, args);
}
My another question is
is it necessary to do clean re-index[delete data directory] after changing the docvalues true? or can we do partially re-index?
Regards,
Vishal Patel
________________________________
From: Vishal Patel <vi...@outlook.com>
Sent: Wednesday, April 17, 2024 5:41 PM
To: users@solr.apache.org <us...@solr.apache.org>
Subject: docValues true is not working for solr.TextField in Solr 8.9.0
We are using Solr 8.9.0. We have configured Solr cloud like 2 shards and each shard has one replica. We have used 7 zoo keepers for Solr cloud.
Our schema field is like
<field name="title" type="text_string" indexed="true" stored="true" multiValued="false" omitNorms="true" termVectors="false" termPositions="false" termOffsets="false" omitTermFreqAndPositions="true"/>
<dynamicField name="AIR_ts_s_*" type="text_string" indexed="true" stored="true" omitTermFreqAndPositions="true"/>
<fieldtype name="text_string" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldtype>
We are using sorting and faceting on both field title and AIR_ts_s_100. We see the both field in fieldCache using admin panel and it is taking much memory. After the long time it is not decreasing.
We know the we can not apply docValues true directly in solr.TextField. So, We have taken reference from https://github.com/grossws/solr-dvtf.
It is working fine for title like after applying sort and facet on title field we can not see in fieldCache but when we are applying sort on AIR_ts_s_100 then we are getting ERROR
org.apache.solr.common.SolrException: can not sort on multivalued field: AIR_ts_s_100 of type: text_string
at org.apache.solr.schema.SchemaField.checkSortability(SchemaField.java:188)
at org.apache.solr.schema.FieldType.getSortedSetSortField(FieldType.java:732)
at org.apache.solr.schema.TextField.getSortField(TextField.java:121)
at org.apache.solr.schema.SchemaField.getSortField(SchemaField.java:154)
Can you please suggest us why we are getting ERROR like can not sort on multivalued field? because it is not multi value true.
Our main issue is day by day increasing the fieldCache size. How can we avoid or reduce size for solr.TextField?
Regards,
Vishal