You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Raj Yadav <ra...@cse.ism.ac.in> on 2020/07/22 12:08:40 UTC
Why External File Field is marked as indexed in solr admin SCHEMA page?
We have following external file field type and field:
*<fieldType name="ext_file_field" keyField="id" defVal="0.0"
class="solr.ExternalFileField" valType="float"/>*
*<field name="fieldA" type="ext_file_field" />*
In solr official documentation is it mentioned that :
*"*The ExternalFileField type makes it possible to specify the values for a
field in a file outside the Solr index. *External fields are not
searchable. They can be used only for function queries or display."*
I was expecting that for field "fieldA" indexed will be marked as false and
it will not be part of the index. But Solr admin "SCHEMA page" (we get this
option after selecting collection name in the drop-down menu) is showing
it as an indexed field (green tick mark under Indexed flag).
We have not explicitly specified indexed=false for this external field in
our schema. Wanted to know whether this field is really part of the index.
Or it is just a bug from the admin UI side.
Regards,
Raj
Re: Why External File Field is marked as indexed in solr admin
SCHEMA page?
Posted by "raj.yadav" <ra...@cse.ism.ac.in>.
Hi Chris,
Chris Hostetter-3 wrote
> ...ExternalFileField is "special" and as noted in it's docs it is not
> searchable -- it doesn't actaully care what the indexed (or "stored")
> properties are ... but the default values of those properties as assigend
> by the schema defaults are still there in the metadata of the field --
> which is what the schema API/browser are showing you.
As you mentioned above, that the `stored` parameter will also be ignored
(i.e doesn't matter whether its marked as false or true). So when we
retrieve the external fields using the `fl = field(exteranl_field_name)`
solr will always retrieve the field value from the external file.
Regards,
Raj
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Why External File Field is marked as indexed in solr admin
SCHEMA page?
Posted by "raj.yadav" <ra...@cse.ism.ac.in>.
Chris Hostetter-3 wrote
> : *
> <fieldType name="ext_file_field" keyField="id" defVal="0.0"
> : class="solr.ExternalFileField" valType="float"/>
> *
> :
> : *
> <field name="fieldA" type="ext_file_field" />
> *
> ...
> : I was expecting that for field "fieldA" indexed will be marked as false
> and
> : it will not be part of the index. But Solr admin "SCHEMA page" (we get
> this
> : option after selecting collection name in the drop-down menu) is
> showing
> : it as an indexed field (green tick mark under Indexed flag).
>
> Because, per the docs, the IndexSchema uses a default assumption of "true"
> for the "indexed" property (if not specified at a field/fieldtype level)
> ...
>
> https://lucene.apache.org/solr/guide/8_4/field-type-definitions-and-properties.html#field-default-properties
>
> Property: indexed
> Descrption: If true, the value of the field can be used in queries to
> retrieve matching documents.
> Values: true or false
> Implicit Default: true
>
> ...ExternalFileField is "special" and as noted in it's docs it is not
> searchable -- it doesn't actaully care what the indexed (or "stored")
> properties are ... but the default values of those properties as assigend
> by the schema defaults are still there in the metadata of the field --
> which is what the schema API/browser are showing you.
>
>
> Imagine you had a a
> <field/>
> that was a TextField -- implicitly
> indexed="true" -- but it was impossible for you to ever put any values
> in that field (say for hte sake of argument you used an analyzier that
> threw away all terms). The schema browser would say: "It's (implicitly)
> marked indexed=true, therefore it's searchable" even though searching on
> that
> field would never return anything ... equivilent situation with
> ExternalFileField.
>
> (ExternalFileField could be modified to override the implicit default for
> these properties, but that's not something anyone has ever really worried
> about because it wouldn't functionally change any of it's behavior)
>
>
> -Hoss
> http://www.lucidworks.com/
Thanks Chris.
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Why External File Field is marked as indexed in solr admin SCHEMA
page?
Posted by Chris Hostetter <ho...@fucit.org>.
: *<fieldType name="ext_file_field" keyField="id" defVal="0.0"
: class="solr.ExternalFileField" valType="float"/>*
:
: *<field name="fieldA" type="ext_file_field" />*
...
: I was expecting that for field "fieldA" indexed will be marked as false and
: it will not be part of the index. But Solr admin "SCHEMA page" (we get this
: option after selecting collection name in the drop-down menu) is showing
: it as an indexed field (green tick mark under Indexed flag).
Because, per the docs, the IndexSchema uses a default assumption of "true"
for the "indexed" property (if not specified at a field/fieldtype level)
...
https://lucene.apache.org/solr/guide/8_4/field-type-definitions-and-properties.html#field-default-properties
Property: indexed
Descrption: If true, the value of the field can be used in queries to retrieve matching documents.
Values: true or false
Implicit Default: true
...ExternalFileField is "special" and as noted in it's docs it is not
searchable -- it doesn't actaully care what the indexed (or "stored")
properties are ... but the default values of those properties as assigend
by the schema defaults are still there in the metadata of the field --
which is what the schema API/browser are showing you.
Imagine you had a a <field/> that was a TextField -- implicitly
indexed="true" -- but it was impossible for you to ever put any values
in that field (say for hte sake of argument you used an analyzier that
threw away all terms). The schema browser would say: "It's (implicitly)
marked indexed=true, therefore it's searchable" even though searching on that
field would never return anything ... equivilent situation with
ExternalFileField.
(ExternalFileField could be modified to override the implicit default for
these properties, but that's not something anyone has ever really worried
about because it wouldn't functionally change any of it's behavior)
-Hoss
http://www.lucidworks.com/
RE: Why External File Field is marked as indexed in solr admin
SCHEMA page?
Posted by "raj.yadav" <ra...@cse.ism.ac.in>.
Vadim Ivanov wrote
> Hello, Raj
>
> I've just checked my Schema page for external file field
>
> Solr version 8.3.1 gives only such parameters for externalFileField:
>
>
> Field: fff
>
> Field-Type:
>
> org.apache.solr.schema.ExternalFileField
>
>
> Flags:
>
> UnInvertible
>
> Omit Term Frequencies & Positions
>
>
> Properties
>
> √
>
> √
>
>
> Are u sure you don’t have (or had) fieldA in main collection schema ?
>
>
>
> externalFileField is not part of the index. It resides in separate file in
> Solr index directory and goes into memory every commit.
Hi Vadim Ivanov,
Earlier the fieldType and field I shared with were from solr_5.4.
I have cross the same thing in solr_8.5.2. I have created the following two
fieldTypes and field.
<fieldType name="ext_file_fieldA" indexed="false" stored="true"
keyField="id" defVal="0" class="solr.ExternalFileField"/>
<fieldType name="ext_file_fieldB" keyField="id" defVal="0"
class="solr.ExternalFileField"/>
<field name="fieldA" type="ext_file_fieldA" />
<field name="fieldB" type="ext_file_fieldB" />
In fieldType `ext_file_fieldA` since I have explicitly mentioned about
indexed and stored parameter. I'm getting the expected result in solr SCHEMA
page. (PFA image file: fieldA_schema)
In fieldType `ext_file_fieldB` not mentioned anything about indexed and
stored parameters. I was expecting that the indexed parameter will be false
by default. But in solr SCHEMA page indexed flag is marked green √ (PFA
image file: fieldB_schema)
Please find attached files.
Regards,
Raj <https://lucene.472066.n3.nabble.com/file/t495721/fieldA_schema.png>
<https://lucene.472066.n3.nabble.com/file/t495721/fieldB_schema.png>
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
RE: Why External File Field is marked as indexed in solr admin SCHEMA page?
Posted by Vadim Ivanov <va...@spb.ntk-intourist.ru>.
Hello, Raj
I've just checked my Schema page for external file field
Solr version 8.3.1 gives only such parameters for externalFileField:
Field: fff
Field-Type:
org.apache.solr.schema.ExternalFileField
Flags:
UnInvertible
Omit Term Frequencies & Positions
Properties
√
√
Are u sure you don’t have (or had) fieldA in main collection schema ?
externalFileField is not part of the index. It resides in separate file in Solr index directory and goes into memory every commit.
> -----Original Message-----
> From: Raj Yadav [mailto:rajkumar1@cse.ism.ac.in]
> Sent: Wednesday, July 22, 2020 3:09 PM
> To: solr-user@lucene.apache.org
> Subject: Why External File Field is marked as indexed in solr admin SCHEMA
> page?
>
> We have following external file field type and field:
>
> *<fieldType name="ext_file_field" keyField="id" defVal="0.0"
> class="solr.ExternalFileField" valType="float"/>*
>
> *<field name="fieldA" type="ext_file_field" />*
>
> In solr official documentation is it mentioned that :
> *"*The ExternalFileField type makes it possible to specify the values for a
> field in a file outside the Solr index. *External fields are not searchable. They
> can be used only for function queries or display."*
>
> I was expecting that for field "fieldA" indexed will be marked as false and it
> will not be part of the index. But Solr admin "SCHEMA page" (we get this
> option after selecting collection name in the drop-down menu) is showing it
> as an indexed field (green tick mark under Indexed flag).
>
> We have not explicitly specified indexed=false for this external field in our
> schema. Wanted to know whether this field is really part of the index.
> Or it is just a bug from the admin UI side.
>
> Regards,
> Raj