You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by txlap786 <tx...@hotmail.com> on 2017/07/04 15:15:16 UTC

xml indexing

Hello everyone o/. Im trying to index a xml file using DIH. 

Its mostly like this.

---- EXAMPLE DIH CONFIG STRUCTURE----
<entity processor=&quot;FileListEntityProcessor&quot; ..
        &lt;entity processor=&quot;XPathEntityProcessor&quot; ..
forEach=&quot;/entryHeader&quot;
                &lt;field column=.. xpath=.. />
                <field column=.. xpath=.. />
                <field column=.. xpath=.. />
        </entity>
</entity>

---- EXAMPLE XML STRUCTURE ----
<gl-cor:entryHeader>                            
	<gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
	<gl-cor:entryNumber> xx </gl-cor:entryNumber>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
		<gl-cor:detailComment> xx </gl-cor:detailComment>
	</gl-cor:entryDetail>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
		<gl-cor:detailComment> xx </gl-cor:detailComment>
	</gl-cor:entryDetail>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
		<gl-cor:detailComment> xx </gl-cor:detailComment>
	</gl-cor:entryDetail>
</gl-cor:entryHeader>
<gl-cor:entryHeader>                            
	<gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
	<gl-cor:entryNumber> xx </gl-cor:entryNumber>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
		<gl-cor:detailComment> xx </gl-cor:detailComment>
	</gl-cor:entryDetail>
</gl-cor:entryHeader>
<gl-cor:entryHeader>                            
	<gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
	<gl-cor:entryNumber> xx </gl-cor:entryNumber>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
	</gl-cor:entryDetail>
	<gl-cor:entryDetail>
		<gl-cor:lineNumber> xx </gl-cor:lineNumber>
	</gl-cor:entryDetail>
</gl-cor:entryHeader>

(at  detailComment doesnt exist) !! 

---- JSON return ----

"detailComment",
[
"100.01",
"102.01",
"102.02",
"120.01",
"120.02",
"153.01",
"320.01",
null,
null
]

---- INDEXED ----

"detailComment" : [
"100.01",
"102.01",
"102.02",
"120.01",
"120.02",
"153.01",
"320.01"  
]


so,
<field name="detailComment" ... multiValued="true" default="somethingelse"/>
default doesnt work due to multivalued

How can i index those null as something visible. like "0","null","NULL" or
"empty"

I want the indexed ones to be same as json return.. 

Can i use xPathprocessor inside of xPathprocessor to get those "entryDetail"
?
So i wont have to use multivalues anymore. just gonna set default values for
each



--
View this message in context: http://lucene.472066.n3.nabble.com/xml-indexing-tp4344191.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: xml indexing

Posted by txlap786 <tx...@hotmail.com>.
Thanks for your reply,
but it only works when i got no response.  But as i said im working on
arrays. As soon as i get an array it doesnt matter if array's length is 1 or
105 it returns what i get earlier.

---- #1 ---- json response
"detailComment", 
[ 
"100.01",
null,
"102.01",
null 
] 
return as
---- #1 ---- indexed
"detailComment": [ 
"100.01",
"102.01"
] 

---- #2 ---- json response
"detailComment", 
[ 
null 
] 
return as
---- #2 ---- indexed
"detailComment": [ 
"0.0"
] 

Result i want to see
---- #3 ---- json response
"detailComment", 
[ 
"100.01",
null,
"102.01",
null 
] 
return as
---- #3 ---- indexed
"detailComment": [ 
"100.01",
"0.0",
"102.01",
"0.0"
] 

<updateProcessor  class="solr.DefaultValueUpdateProcessorFactory"
name="upd">
	<str name="fieldName">detailComment</str>
	<float name="value">0.0</float>
</updateProcessor>

<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
	<lst name="defaults">
		<str name="config">dih-config.xml</str>
		<str name="processor">upd</str>
	</lst>
</requestHandler>



--
View this message in context: http://lucene.472066.n3.nabble.com/xml-indexing-tp4344191p4344298.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: xml indexing

Posted by Alexandre Rafalovitch <ar...@gmail.com>.
You can set default values in the UpdateRequestProcessor chain:
http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/update/processor/DefaultValueUpdateProcessorFactory.html

You can combine URPs with DIH. There is an example for that in the latest Solr:
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/example/example-DIH/solr/atom/conf/solrconfig.xml

Regards,
   Alex.
----
http://www.solr-start.com/ - Resources for Solr users, new and experienced


On 4 July 2017 at 11:15, txlap786 <tx...@hotmail.com> wrote:
> Hello everyone o/. Im trying to index a xml file using DIH.
>
> Its mostly like this.
>
> ---- EXAMPLE DIH CONFIG STRUCTURE----
> <entity processor="FileListEntityProcessor" ..
>         <entity processor="XPathEntityProcessor" ..
> forEach="/entryHeader"
>                 <field column=.. xpath=.. />
>                 <field column=.. xpath=.. />
>                 <field column=.. xpath=.. />
>         </entity>
> </entity>
>
> ---- EXAMPLE XML STRUCTURE ----
> <gl-cor:entryHeader>
>         <gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
>         <gl-cor:entryNumber> xx </gl-cor:entryNumber>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>                 <gl-cor:detailComment> xx </gl-cor:detailComment>
>         </gl-cor:entryDetail>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>                 <gl-cor:detailComment> xx </gl-cor:detailComment>
>         </gl-cor:entryDetail>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>                 <gl-cor:detailComment> xx </gl-cor:detailComment>
>         </gl-cor:entryDetail>
> </gl-cor:entryHeader>
> <gl-cor:entryHeader>
>         <gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
>         <gl-cor:entryNumber> xx </gl-cor:entryNumber>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>                 <gl-cor:detailComment> xx </gl-cor:detailComment>
>         </gl-cor:entryDetail>
> </gl-cor:entryHeader>
> <gl-cor:entryHeader>
>         <gl-cor:entryNumberCounter> xx </gl-cor:entryNumberCounter>
>         <gl-cor:entryNumber> xx </gl-cor:entryNumber>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>         </gl-cor:entryDetail>
>         <gl-cor:entryDetail>
>                 <gl-cor:lineNumber> xx </gl-cor:lineNumber>
>         </gl-cor:entryDetail>
> </gl-cor:entryHeader>
>
> (at  detailComment doesnt exist) !!
>
> ---- JSON return ----
>
> "detailComment",
> [
> "100.01",
> "102.01",
> "102.02",
> "120.01",
> "120.02",
> "153.01",
> "320.01",
> null,
> null
> ]
>
> ---- INDEXED ----
>
> "detailComment" : [
> "100.01",
> "102.01",
> "102.02",
> "120.01",
> "120.02",
> "153.01",
> "320.01"
> ]
>
>
> so,
> <field name="detailComment" ... multiValued="true" default="somethingelse"/>
> default doesnt work due to multivalued
>
> How can i index those null as something visible. like "0","null","NULL" or
> "empty"
>
> I want the indexed ones to be same as json return..
>
> Can i use xPathprocessor inside of xPathprocessor to get those "entryDetail"
> ?
> So i wont have to use multivalues anymore. just gonna set default values for
> each
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/xml-indexing-tp4344191.html
> Sent from the Solr - User mailing list archive at Nabble.com.