You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Triveni <tr...@gmail.com> on 2017/12/15 10:07:01 UTC

SOLR nested dataimport issues

I am trying to import a nested xml using URLdatasource. But indexing is not
happening.
XML:
<hash>
<name>ABC</name>
<createdAt>1512016450886</createdAt>
<createdBy>XYZ</createdBy>
<xyz type="array">
<xyz>
<name>access</name>
<value>public</value>
</xyz>
<xyz>
<name>access12</name>
<value>public12</value>
</xyz>
</xyz>
</hash>

My data-config.xml:
<dataConfig>

  <dataSource type="URLDataSource"/>
  <document>
    <entity name="sample"
            url="http://abc:123/api/sample_api.xml"
			processor="XPathEntityProcessor" 
			forEach="/hash" >
    <field column="id" name="id" xpath="/hash/name"/> 
	  <field column="createdBy" name="createdBy" xpath="/hash/createdBy"/>
	  	   <entity child="true" name="sam" join="zipper"
            url="http://abc:123/api/sample_api.xml"
			processor="XPathEntityProcessor" forEach="/hash/xyz/xyz"
transformer="script:f1">
		<field column="attrName" name="attr.attrName" xpath="/hash/xyz/xyz/name"/>
	   <field column="attrValue" name="attr.attrValue"
xpath="/hash/xyz/xyz/value"/>
	   </entity>
	   </entity>
  </document>
</dataConfig> 

I am seeing below message when indexing:
Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
Requests: 2 , Fetched: 3 , Skipped: 0 , Processed: 0 
Started: less than a minute ago

And in solr.log file below error:
2017-12-15 09:38:53.254 WARN  (qtp223684-13) [   x:xml_data]
o.a.s.h.d.SolrWriter Error creating document : SolrInputDocument(fields:
[createdBy=XYZ, id=ABC, _version_=1586842286933671936, _root_=ABC],
children: [SolrInputDocument(fields: [attr.attrValue=public, attr.Id=1,
attr.attrName=access, _root_=ABC, _version_=1586842286933671936]),
SolrInputDocument(fields: [attr.attrValue=public12, attr.Id=2,
attr.attrName=access12, _root_=ABC, _version_=1586842286933671936])])
org.apache.solr.common.SolrException: [doc=null] missing required field: id
	at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:265)
	at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:107)



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR nested dataimport issues

Posted by Emir Arnautović <em...@sematext.com>.
Hi,
I did not check it but it seems to me that it might be related to using full path in your fields xpath: you are iterating hash-es and you should probable set field paths assuming it is the new root. E.g. for id it would be:
> <field column=“id" name="id" xpath="/name"/>

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 15 Dec 2017, at 11:07, Triveni <tr...@gmail.com> wrote:
> 
> I am trying to import a nested xml using URLdatasource. But indexing is not
> happening.
> XML:
> <hash>
> <name>ABC</name>
> <createdAt>1512016450886</createdAt>
> <createdBy>XYZ</createdBy>
> <xyz type="array">
> <xyz>
> <name>access</name>
> <value>public</value>
> </xyz>
> <xyz>
> <name>access12</name>
> <value>public12</value>
> </xyz>
> </xyz>
> </hash>
> 
> My data-config.xml:
> <dataConfig>
> 
>  <dataSource type="URLDataSource"/>
>  <document>
>    <entity name="sample"
>            url="http://abc:123/api/sample_api.xml"
> 			processor="XPathEntityProcessor" 
> 			forEach="/hash" >
>    <field column="id" name="id" xpath="/hash/name"/> 
> 	  <field column="createdBy" name="createdBy" xpath="/hash/createdBy"/>
> 	  	   <entity child="true" name="sam" join="zipper"
>            url="http://abc:123/api/sample_api.xml"
> 			processor="XPathEntityProcessor" forEach="/hash/xyz/xyz"
> transformer="script:f1">
> 		<field column="attrName" name="attr.attrName" xpath="/hash/xyz/xyz/name"/>
> 	   <field column="attrValue" name="attr.attrValue"
> xpath="/hash/xyz/xyz/value"/>
> 	   </entity>
> 	   </entity>
>  </document>
> </dataConfig> 
> 
> I am seeing below message when indexing:
> Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
> Requests: 2 , Fetched: 3 , Skipped: 0 , Processed: 0 
> Started: less than a minute ago
> 
> And in solr.log file below error:
> 2017-12-15 09:38:53.254 WARN  (qtp223684-13) [   x:xml_data]
> o.a.s.h.d.SolrWriter Error creating document : SolrInputDocument(fields:
> [createdBy=XYZ, id=ABC, _version_=1586842286933671936, _root_=ABC],
> children: [SolrInputDocument(fields: [attr.attrValue=public, attr.Id=1,
> attr.attrName=access, _root_=ABC, _version_=1586842286933671936]),
> SolrInputDocument(fields: [attr.attrValue=public12, attr.Id=2,
> attr.attrName=access12, _root_=ABC, _version_=1586842286933671936])])
> org.apache.solr.common.SolrException: [doc=null] missing required field: id
> 	at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:265)
> 	at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:107)
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html