You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jeebix <jb...@initiatives.fr> on 2018/07/02 15:18:12 UTC
_childDocuments_ automatically multivalued field type
Hello everybody,
I have a problem with some field types in the managed-schema generated.
First, the data SOLR returned with a standard query :
response":{"numFound":365567,"start":0,"docs":[
{
"id":"560.561.134676",
"parent_i":560,
"asso_i":561,
"personne_i":134676,
"etat_technique_s":"avec_documents",
"etat_marketing_s":"actif",
"type_parent_s":"Ecole élémentaire publique",
"type_asso_s":"APE (association de parents d'élèves)",
"groupe_type_parent_s":"ENSEIGNEMENT_PRIMAIRE",
"groupe_type_asso_s":"ASSOCIATION_DE_PARENTS",
"nombre_commandes_brut_i":2,
"nombre_commandes_i":1,
"nombre_kits_saveur_i":0,
"ca_periode_i":560,
"ca_periode_fleur_i":0,
"ca_periode_saveur_i":0,
"zone_scolaire_s":"A",
"territoire_s":"France Métropolitaine",
"region_s":"AUVERGNE RHONE-ALPES",
"departement_s":"01 AIN",
"postal_country_s":"FR",
"asso_country_s":"FRANCE",
"object_type_s":"contact",
"date_derni_re_commande_dt":"2016-05-20T00:00:00Z",
"_version_":1604889647955050496,
"_childDocuments_":[
{
"fixe_facturation":["0256897856"],
"object_type":["order"],
"mobile_livraison":["0658987874"],
"kit_sans_suite":["false"],
"fixe_livraison":["0450598311"],
"type_cde_s":"CDE",
"statut_s":"V",
"mobile_facturation":["0658787458"],
"campagne_s":"A",
"TTC":[780],
"date_dt":"2016-05-20T00:00:00Z",
"id":"A28837",
"enseigne_s":"CRE"},
{
"fixe_facturation":["0245784975"],
"object_type":["order"],
"mobile_livraison":["0645789874"],
"kit_sans_suite":["false"],
"type_cde_s":"KIT",
"statut_s":"V",
"mobile_facturation":["0612345678"],
"campagne_s":"A",
"TTC":[0],
"date_dt":"2016-05-04T00:00:00Z",
"id":"A25415",
"enseigne_s":"CRE"}]}
My goal is to sum fields "TTC" by parentDocument. But with the type
"multiValued", I can't use aggregation functions.
The core get the data from this script : /opt/solr/bin/post -c <core_name>
-format solr build/index.json
The index.json looks like that:
[
{
"id": "781.782.134878",
"parent_i": 781,
"asso_i": 782,
"personne_i": 134878,
"etat_technique_s": "avec_documents",
"etat_marketing_s": "inactif",
"type_parent_s": "Ecole élémentaire privée",
"type_asso_s": "APEL (association de parents école libre)",
"groupe_type_parent_s": "ENSEIGNEMENT_PRIMAIRE",
"groupe_type_asso_s": "ASSOCIATION_DE_PARENTS",
"nombre_commandes_brut_i": 4,
"nombre_commandes_i": 2,
"nombre_kits_saveur_i": 2,
"date_dernière_commande_dt": "2010-11-16",
"ca_periode_i": 0,
"ca_periode_fleur_i": 0,
"ca_periode_saveur_i": 0,
"zone_scolaire_s": "A",
"territoire_s": "France Métropolitaine",
"region_s": "AUVERGNE RHONE-ALPES",
"departement_s": "01 AIN",
"postal_country_s": "FR",
"asso_country_s": "FRANCE",
"object_type_s": "contact",
"kits_sans_suite_ss": null,
"_childDocuments_": [
{
"fixe_facturation": "0450407279",
"object_type": "order",
"mobile_livraison": "0628332864",
"kit_sans_suite": "false",
"fixe_livraison": "0450407279",
"type_cde_s": "KIT",
"statut_s": "V",
"mobile_facturation": "0628332864",
"campagne_s": "L",
"TTC": 0,
"date_dt": "2009-10-12T00:00:00Z",
"id": "L14276",
"enseigne_s": "SAV",
"gamme": [
"KITS > Kits Saveurs"
]
},
{
"fixe_facturation": "0450407279",
"object_type": "order",
"mobile_livraison": "0628332864",
"kit_sans_suite": "false",
"fixe_livraison": "0450407279",
"type_cde_s": "CDE",
"statut_s": "V",
"mobile_facturation": "0628332864",
"campagne_s": "L",
"TTC": 1045,
"date_dt": "2009-11-14T00:00:00Z",
"id": "L25049",
"enseigne_s": "SAV",
"gamme": [
"CHOCOLAT > Assortiment",
"CHOCOLAT > Individuel",
"CHOCOLAT > Mono-produit",
"EQUIPEMENT MAISON > Cuisine",
"EQUIPEMENT MAISON > Décoration",
"KITS > Kits Saveurs",
"SAVEURS > Confiserie",
"SAVEURS > Pâtisserie"
]
}
]
In the managed-schema, only those fields appear:
<field name="TTC" type="tlongs"/>
<field name="_root_" type="string" docValues="false" indexed="true"
stored="false"/>
<field name="_text_" type="text_general" multiValued="true" indexed="true"
stored="false"/>
<field name="_version_" type="long" indexed="true" stored="false"/>
<field name="fixe_facturation" type="strings"/>
<field name="fixe_livraison" type="strings"/>
<field name="gamme" type="strings"/>
<field name="id" type="string" multiValued="false" indexed="true"
required="true" stored="true"/>
<field name="kit_sans_suite" type="strings"/>
<field name="mobile_facturation" type="strings"/>
<field name="mobile_livraison" type="strings"/>
<field name="object_type" type="strings"/>
I don't understand why for example "type_cmd_s" get the field type attribute
"singleValued", but "TTC" or "kits_sans_suite" get "multiValued" attribute ?
Why those field are in the managed-schema and enseigne_s (for example) is
not ?
Thanks a lot for your help...
Best
JB
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: _childDocuments_ automatically multivalued field type
Posted by jeebix <jb...@initiatives.fr>.
Ok, I'll have a look at the link above.
Thanks a lot...
Best
JB
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: _childDocuments_ automatically multivalued field type
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/2/2018 9:18 AM, jeebix wrote:
> I don't understand why for example "type_cmd_s" get the field type attribute
> "singleValued", but "TTC" or "kits_sans_suite" get "multiValued" attribute ?
> Why those field are in the managed-schema and enseigne_s (for example) is
> not ?
The field named enseigne_s is almost certainly handled by a dynamic
field definition, most likely one with the name "*_s". That field (and
its field type) do not have multiValued="true". This was probably
already in your schema before you did any indexing.
The ones that were automatically added by the data-driven nature of your
schema were added as the "strings" type, which IS multi-valued. The
update processor definition that is in the Solr examples is set up to
add fields as multiValued, so that if a later indexing request comes in
with multiple values for the field, it will not fail.
This is the major danger of relying on Solr to automatically add fields
to your schema. Chances are good that the choice it makes for the field
will be the wrong choice. And when that happens, you will need to fix
the schema and completely reindex.
https://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Re: _childDocuments_ automatically multivalued field type
Posted by jeebix <jb...@initiatives.fr>.
Ok, I see what I have to look for, thanks to your reply. I'll adjust the
schema and see difference.
Thanks.
Best
JB
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: _childDocuments_ automatically multivalued field type
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Because your _s fields must be mapping to the dynamicField definition and
are created accordingly in the schema dynamically without needing a special
definition for each field.
The TTC field you did map explicitly, perhaps with "schemaless" mapping
autodiscovery. Which does create specific field definitions, but always
multivalued.
The multivalued attribute can be set on type not just on individual field.
So you may just want to adjust schema definition to use singular types
instead.
AdminUI schema screen is helpful to see such differences.
Regards,
Alex
On Mon, Jul 2, 2018, 11:43 AM jeebix, <jb...@initiatives.fr> wrote:
> Hello everybody,
>
> I have a problem with some field types in the managed-schema generated.
>
> First, the data SOLR returned with a standard query :
>
> response":{"numFound":365567,"start":0,"docs":[
> {
> "id":"560.561.134676",
> "parent_i":560,
> "asso_i":561,
> "personne_i":134676,
> "etat_technique_s":"avec_documents",
> "etat_marketing_s":"actif",
> "type_parent_s":"Ecole élémentaire publique",
> "type_asso_s":"APE (association de parents d'élèves)",
> "groupe_type_parent_s":"ENSEIGNEMENT_PRIMAIRE",
> "groupe_type_asso_s":"ASSOCIATION_DE_PARENTS",
> "nombre_commandes_brut_i":2,
> "nombre_commandes_i":1,
> "nombre_kits_saveur_i":0,
> "ca_periode_i":560,
> "ca_periode_fleur_i":0,
> "ca_periode_saveur_i":0,
> "zone_scolaire_s":"A",
> "territoire_s":"France Métropolitaine",
> "region_s":"AUVERGNE RHONE-ALPES",
> "departement_s":"01 AIN",
> "postal_country_s":"FR",
> "asso_country_s":"FRANCE",
> "object_type_s":"contact",
> "date_derni_re_commande_dt":"2016-05-20T00:00:00Z",
> "_version_":1604889647955050496,
> "_childDocuments_":[
> {
> "fixe_facturation":["0256897856"],
> "object_type":["order"],
> "mobile_livraison":["0658987874"],
> "kit_sans_suite":["false"],
> "fixe_livraison":["0450598311"],
> "type_cde_s":"CDE",
> "statut_s":"V",
> "mobile_facturation":["0658787458"],
> "campagne_s":"A",
> "TTC":[780],
> "date_dt":"2016-05-20T00:00:00Z",
> "id":"A28837",
> "enseigne_s":"CRE"},
> {
> "fixe_facturation":["0245784975"],
> "object_type":["order"],
> "mobile_livraison":["0645789874"],
> "kit_sans_suite":["false"],
> "type_cde_s":"KIT",
> "statut_s":"V",
> "mobile_facturation":["0612345678"],
> "campagne_s":"A",
> "TTC":[0],
> "date_dt":"2016-05-04T00:00:00Z",
> "id":"A25415",
> "enseigne_s":"CRE"}]}
>
> My goal is to sum fields "TTC" by parentDocument. But with the type
> "multiValued", I can't use aggregation functions.
>
> The core get the data from this script : /opt/solr/bin/post -c <core_name>
> -format solr build/index.json
>
> The index.json looks like that:
>
> [
> {
> "id": "781.782.134878",
> "parent_i": 781,
> "asso_i": 782,
> "personne_i": 134878,
> "etat_technique_s": "avec_documents",
> "etat_marketing_s": "inactif",
> "type_parent_s": "Ecole élémentaire privée",
> "type_asso_s": "APEL (association de parents école libre)",
> "groupe_type_parent_s": "ENSEIGNEMENT_PRIMAIRE",
> "groupe_type_asso_s": "ASSOCIATION_DE_PARENTS",
> "nombre_commandes_brut_i": 4,
> "nombre_commandes_i": 2,
> "nombre_kits_saveur_i": 2,
> "date_dernière_commande_dt": "2010-11-16",
> "ca_periode_i": 0,
> "ca_periode_fleur_i": 0,
> "ca_periode_saveur_i": 0,
> "zone_scolaire_s": "A",
> "territoire_s": "France Métropolitaine",
> "region_s": "AUVERGNE RHONE-ALPES",
> "departement_s": "01 AIN",
> "postal_country_s": "FR",
> "asso_country_s": "FRANCE",
> "object_type_s": "contact",
> "kits_sans_suite_ss": null,
> "_childDocuments_": [
> {
> "fixe_facturation": "0450407279",
> "object_type": "order",
> "mobile_livraison": "0628332864",
> "kit_sans_suite": "false",
> "fixe_livraison": "0450407279",
> "type_cde_s": "KIT",
> "statut_s": "V",
> "mobile_facturation": "0628332864",
> "campagne_s": "L",
> "TTC": 0,
> "date_dt": "2009-10-12T00:00:00Z",
> "id": "L14276",
> "enseigne_s": "SAV",
> "gamme": [
> "KITS > Kits Saveurs"
> ]
> },
> {
> "fixe_facturation": "0450407279",
> "object_type": "order",
> "mobile_livraison": "0628332864",
> "kit_sans_suite": "false",
> "fixe_livraison": "0450407279",
> "type_cde_s": "CDE",
> "statut_s": "V",
> "mobile_facturation": "0628332864",
> "campagne_s": "L",
> "TTC": 1045,
> "date_dt": "2009-11-14T00:00:00Z",
> "id": "L25049",
> "enseigne_s": "SAV",
> "gamme": [
> "CHOCOLAT > Assortiment",
> "CHOCOLAT > Individuel",
> "CHOCOLAT > Mono-produit",
> "EQUIPEMENT MAISON > Cuisine",
> "EQUIPEMENT MAISON > Décoration",
> "KITS > Kits Saveurs",
> "SAVEURS > Confiserie",
> "SAVEURS > Pâtisserie"
> ]
> }
> ]
>
> In the managed-schema, only those fields appear:
>
> <field name="TTC" type="tlongs"/>
> <field name="_root_" type="string" docValues="false" indexed="true"
> stored="false"/>
> <field name="_text_" type="text_general" multiValued="true" indexed="true"
> stored="false"/>
> <field name="_version_" type="long" indexed="true" stored="false"/>
> <field name="fixe_facturation" type="strings"/>
> <field name="fixe_livraison" type="strings"/>
> <field name="gamme" type="strings"/>
> <field name="id" type="string" multiValued="false" indexed="true"
> required="true" stored="true"/>
> <field name="kit_sans_suite" type="strings"/>
> <field name="mobile_facturation" type="strings"/>
> <field name="mobile_livraison" type="strings"/>
> <field name="object_type" type="strings"/>
>
> I don't understand why for example "type_cmd_s" get the field type
> attribute
> "singleValued", but "TTC" or "kits_sans_suite" get "multiValued" attribute
> ?
> Why those field are in the managed-schema and enseigne_s (for example) is
> not ?
>
> Thanks a lot for your help...
>
> Best
> JB
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>