You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Todd Breiholz <ta...@gmail.com> on 2008/08/20 22:42:12 UTC
DIH - Document missing required field error
I am testing out DataImportHandler (Nightly 08-18-2008) in our environment
with a simple schema. One database view with 10 columns mapped to an index
in Solr. When running the full-import command, I get the error below for
each row. I'm sure I'm missing something, but I can't figure it out.
WARNING: Error creating document : SolrInputDocumnt[{}]
org.apache.solr.common.SolrException: Document [null] missing required
field: id
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
at
org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
at
org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
at
org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
at
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
Here is my data-config.xml:
<dataConfig>
<dataSource type="JdbcDataSource"
driver="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
<document name="sharemy">
<entity name="gallery" pk="mygallery_id"
query="select * from mdp_share_my_search_view">
<field column="mygallery_id" name="id"/>
<field column="mygallrychanl_id" name="channelId"/>
<field column="mygallrycat_id" name="categoryId"/>
<field column="user_id" name="userId"/>
<field column="mygallry_name" name="galleryName"/>
<field column="photo_names" name="photoNames"/>
<field column="photo_descs" name="photoDescriptions"/>
<field column="hit_count" name="hitCount"/>
<field column="vote_count" name="voteCount"/>
<field column="rating" name="rating"/>
<field column="photo_tags" name="photoTags"/>
</entity>
</document>
</dataConfig>
and from my schema.xml:
<fields>
<field name="id" type="string" indexed="true" stored="true"
required="true" />
<field name="channelId" type="string" indexed="true" stored="true"
omitNorms="true"/>
<field name="categoryId" type="string" indexed="true" stored="true"
omitNorms="true"/>
<field name="userId" type="string" indexed="true" stored="true"
omitNorms="true"/>
<field name="galleryName" type="text" indexed="true" stored="true"/>
<field name="photoNames" type="text" indexed="true" stored="true"/>
<field name="photoDescriptions" type="text" indexed="true"
stored="true"/>
<field name="hitCount" type="sint" indexed="true" stored="true"/>
<field name="voteCount" type="sint" indexed="true" stored="true"/>
<field name="rating" type="sfloat" indexed="true" stored="true"/>
<field name="photoTags" type="text" indexed="true" stored="true"/>
<field name="allText" type="text" indexed="true" stored="false"
multiValued="true"/>
</fields>
<uniqueKey>id</uniqueKey>
Re: DIH - Document missing required field error
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hmm...seems like none of the columns are getting copied over to the
document.
I see that all column names are in caps in Row#1, perhaps you need to add
them in caps to the data-config too? DIH just stores each row as a
Map<String, String> and therefore map lookups are also case-sensitive.
On Thu, Aug 21, 2008 at 7:33 PM, Todd Breiholz <ta...@gmail.com> wrote:
> On Wed, Aug 20, 2008 at 10:54 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
> > Does the "mygallery_id" column in your database allow nulls? It gets
> copied
> > to the id field in Solr which is required, hence the error.
> >
>
> Nope. I just confirmed that the mygallery_id column does *not* allow nulls.
> In fact, the debug output from the /dataimport request shows:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">579</int>
> </lst>
> <lst name="initArgs">
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst></lst><str name="command">full-import</str>
> <str name="mode">debug</str>
> <arr name="documents">
> <arr/>
> </arr>
> <lst name="verbose-output">
> <lst name="entity:gallery">
> <lst name="document#1">
> <str name="query">select * from mdp_share_my_search_view
> where mygallery_id = '1301801'</str>
> <str name="time-taken">0:0:0.188</str>
> <str>----------- row #1-------------</str>
> <str name="MYGALLRY_NAME">Princess Party</str>
> <str name="USER_ID">162900018</str>
> <str name="PHOTO_NAMES">Princessparty</str>
> <str name="MYGALLERY_ID">1301801</str>
> <str name="PHOTO_TAGS">Made in Minutes,Photography,Paper and
> cardstockMade in Minutes,SBE Inspired,Embellishing,Cards,Paper and
> cardstock,Paper and cardstockEmbellishing,Cards,Paper and
> cardstock,Paper and cardsto</str>
> <str name="MYGALLRYCHANL_ID">900002</str>
> <str name="HIT_COUNT">java.math.BigDecimal:0</str>
> <str name="MYGALLRYCAT_ID">cat3800005dev</str>
> <null name="RATING"/>
> <str name="PHOTO_DESCS">Saylors 3rd & Sarahs 1st
> birthday were celebrated in style. We had so much fun celebrating our
> princesses birthdays.</str>
> <str name="VOTE_COUNT">java.math.BigDecimal:0</str>
> <str>---------------------------------------------</str>
> </lst>
> <lst name="document#1"/>
> </lst>
> </lst>
> <str name="status">idle</str>
> <str name="importResponse">Configuration Re-loaded sucessfully</str>
> <lst name="statusMessages">
> <str name="Total Requests made to DataSource">1</str>
> <str name="Total Rows Fetched">1</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2008-08-21 08:55:21</str>
> <str name="">Indexing completed. Added/Updated: 0 documents.
> Deleted 0 documents.</str>
> <str name="Committed">2008-08-21 08:55:21</str>
> <str name="Optimized">2008-08-21 08:55:21</str>
> <str name="Time taken ">0:0:0.563</str>
> </lst>
> <str name="WARNING">This response format is experimental. It is
> likely to change in the future.</str>
> </response>
>
>
>
>
> >
> > On Thu, Aug 21, 2008 at 2:12 AM, Todd Breiholz <ta...@gmail.com> wrote:
> >
> > > I am testing out DataImportHandler (Nightly 08-18-2008) in our
> > environment
> > > with a simple schema. One database view with 10 columns mapped to an
> > index
> > > in Solr. When running the full-import command, I get the error below
> for
> > > each row. I'm sure I'm missing something, but I can't figure it out.
> > >
> > > WARNING: Error creating document : SolrInputDocumnt[{}]
> > > org.apache.solr.common.SolrException: Document [null] missing required
> > > field: id
> > > at
> > >
> >
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
> > > at
> > >
> > >
> >
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
> > > at
> > >
> org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
> > > at
> > >
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
> > > at
> > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
> > > at
> > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> > > at
> > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> > > at
> > >
> > >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > > at
> > >
> > >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > > at
> > >
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> > > at
> > >
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
> > > at
> > >
> > >
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> > > at
> > > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> > > at java.lang.Thread.run(Thread.java:619)
> > >
> > > Here is my data-config.xml:
> > >
> > > <dataConfig>
> > > <dataSource type="JdbcDataSource"
> > > driver="oracle.jdbc.driver.OracleDriver"
> > > url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
> > > <document name="sharemy">
> > > <entity name="gallery" pk="mygallery_id"
> > > query="select * from mdp_share_my_search_view">
> > > <field column="mygallery_id" name="id"/>
> > > <field column="mygallrychanl_id" name="channelId"/>
> > > <field column="mygallrycat_id" name="categoryId"/>
> > > <field column="user_id" name="userId"/>
> > > <field column="mygallry_name" name="galleryName"/>
> > > <field column="photo_names" name="photoNames"/>
> > > <field column="photo_descs" name="photoDescriptions"/>
> > > <field column="hit_count" name="hitCount"/>
> > > <field column="vote_count" name="voteCount"/>
> > > <field column="rating" name="rating"/>
> > > <field column="photo_tags" name="photoTags"/>
> > > </entity>
> > > </document>
> > > </dataConfig>
> > >
> > > and from my schema.xml:
> > >
> > > <fields>
> > > <field name="id" type="string" indexed="true" stored="true"
> > > required="true" />
> > > <field name="channelId" type="string" indexed="true" stored="true"
> > > omitNorms="true"/>
> > > <field name="categoryId" type="string" indexed="true" stored="true"
> > > omitNorms="true"/>
> > > <field name="userId" type="string" indexed="true" stored="true"
> > > omitNorms="true"/>
> > > <field name="galleryName" type="text" indexed="true" stored="true"/>
> > > <field name="photoNames" type="text" indexed="true" stored="true"/>
> > > <field name="photoDescriptions" type="text" indexed="true"
> > > stored="true"/>
> > > <field name="hitCount" type="sint" indexed="true" stored="true"/>
> > > <field name="voteCount" type="sint" indexed="true" stored="true"/>
> > > <field name="rating" type="sfloat" indexed="true" stored="true"/>
> > > <field name="photoTags" type="text" indexed="true" stored="true"/>
> > > <field name="allText" type="text" indexed="true" stored="false"
> > > multiValued="true"/>
> > > </fields>
> > > <uniqueKey>id</uniqueKey>
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
--
Regards,
Shalin Shekhar Mangar.
Re: DIH - Document missing required field error
Posted by Todd Breiholz <ta...@gmail.com>.
That was it. Thanks!
On Thu, Aug 21, 2008 at 9:30 AM, Noble Paul നോബിള് नोब्ळ् <
noble.paul@gmail.com> wrote:
> oh the names are in CAPS. And in your dataconfig it is in small it
> must be something like this
> <field column="MYGALLERY_ID" name="id"/>
>
>
>
> On Thu, Aug 21, 2008 at 7:33 PM, Todd Breiholz <ta...@gmail.com> wrote:
> > On Wed, Aug 20, 2008 at 10:54 PM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> >
> >> Does the "mygallery_id" column in your database allow nulls? It gets
> copied
> >> to the id field in Solr which is required, hence the error.
> >>
> >
> > Nope. I just confirmed that the mygallery_id column does *not* allow
> nulls.
> > In fact, the debug output from the /dataimport request shows:
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <response>
> > <lst name="responseHeader">
> > <int name="status">0</int>
> > <int name="QTime">579</int>
> > </lst>
> > <lst name="initArgs">
> > <lst name="defaults">
> > <str name="config">data-config.xml</str>
> > </lst></lst><str name="command">full-import</str>
> > <str name="mode">debug</str>
> > <arr name="documents">
> > <arr/>
> > </arr>
> > <lst name="verbose-output">
> > <lst name="entity:gallery">
> > <lst name="document#1">
> > <str name="query">select * from mdp_share_my_search_view
> > where mygallery_id = '1301801'</str>
> > <str name="time-taken">0:0:0.188</str>
> > <str>----------- row #1-------------</str>
> > <str name="MYGALLRY_NAME">Princess Party</str>
> > <str name="USER_ID">162900018</str>
> > <str name="PHOTO_NAMES">Princessparty</str>
> > <str name="MYGALLERY_ID">1301801</str>
> > <str name="PHOTO_TAGS">Made in Minutes,Photography,Paper and
> > cardstockMade in Minutes,SBE Inspired,Embellishing,Cards,Paper and
> > cardstock,Paper and cardstockEmbellishing,Cards,Paper and
> > cardstock,Paper and cardsto</str>
> > <str name="MYGALLRYCHANL_ID">900002</str>
> > <str name="HIT_COUNT">java.math.BigDecimal:0</str>
> > <str name="MYGALLRYCAT_ID">cat3800005dev</str>
> > <null name="RATING"/>
> > <str name="PHOTO_DESCS">Saylors 3rd & Sarahs 1st
> > birthday were celebrated in style. We had so much fun celebrating our
> > princesses birthdays.</str>
> > <str name="VOTE_COUNT">java.math.BigDecimal:0</str>
> > <str>---------------------------------------------</str>
> > </lst>
> > <lst name="document#1"/>
> > </lst>
> > </lst>
> > <str name="status">idle</str>
> > <str name="importResponse">Configuration Re-loaded sucessfully</str>
> > <lst name="statusMessages">
> > <str name="Total Requests made to DataSource">1</str>
> > <str name="Total Rows Fetched">1</str>
> > <str name="Total Documents Skipped">0</str>
> > <str name="Full Dump Started">2008-08-21 08:55:21</str>
> > <str name="">Indexing completed. Added/Updated: 0 documents.
> > Deleted 0 documents.</str>
> > <str name="Committed">2008-08-21 08:55:21</str>
> > <str name="Optimized">2008-08-21 08:55:21</str>
> > <str name="Time taken ">0:0:0.563</str>
> > </lst>
> > <str name="WARNING">This response format is experimental. It is
> > likely to change in the future.</str>
> > </response>
> >
> >
> >
> >
> >>
> >> On Thu, Aug 21, 2008 at 2:12 AM, Todd Breiholz <ta...@gmail.com>
> wrote:
> >>
> >> > I am testing out DataImportHandler (Nightly 08-18-2008) in our
> >> environment
> >> > with a simple schema. One database view with 10 columns mapped to an
> >> index
> >> > in Solr. When running the full-import command, I get the error below
> for
> >> > each row. I'm sure I'm missing something, but I can't figure it out.
> >> >
> >> > WARNING: Error creating document : SolrInputDocumnt[{}]
> >> > org.apache.solr.common.SolrException: Document [null] missing required
> >> > field: id
> >> > at
> >> >
> >>
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
> >> > at
> >> >
> org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
> >> > at
> >> >
> >>
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> >> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> >> > at
> >> >
> >> >
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >> > at
> >> >
> >> >
> >>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >> > at
> >> >
> >>
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> >> > at
> >> >
> >>
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
> >> > at
> >> >
> >> >
> >>
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> >> > at
> >> >
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> >> > at java.lang.Thread.run(Thread.java:619)
> >> >
> >> > Here is my data-config.xml:
> >> >
> >> > <dataConfig>
> >> > <dataSource type="JdbcDataSource"
> >> > driver="oracle.jdbc.driver.OracleDriver"
> >> > url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
> >> > <document name="sharemy">
> >> > <entity name="gallery" pk="mygallery_id"
> >> > query="select * from mdp_share_my_search_view">
> >> > <field column="mygallery_id" name="id"/>
> >> > <field column="mygallrychanl_id" name="channelId"/>
> >> > <field column="mygallrycat_id" name="categoryId"/>
> >> > <field column="user_id" name="userId"/>
> >> > <field column="mygallry_name" name="galleryName"/>
> >> > <field column="photo_names" name="photoNames"/>
> >> > <field column="photo_descs" name="photoDescriptions"/>
> >> > <field column="hit_count" name="hitCount"/>
> >> > <field column="vote_count" name="voteCount"/>
> >> > <field column="rating" name="rating"/>
> >> > <field column="photo_tags" name="photoTags"/>
> >> > </entity>
> >> > </document>
> >> > </dataConfig>
> >> >
> >> > and from my schema.xml:
> >> >
> >> > <fields>
> >> > <field name="id" type="string" indexed="true" stored="true"
> >> > required="true" />
> >> > <field name="channelId" type="string" indexed="true" stored="true"
> >> > omitNorms="true"/>
> >> > <field name="categoryId" type="string" indexed="true" stored="true"
> >> > omitNorms="true"/>
> >> > <field name="userId" type="string" indexed="true" stored="true"
> >> > omitNorms="true"/>
> >> > <field name="galleryName" type="text" indexed="true" stored="true"/>
> >> > <field name="photoNames" type="text" indexed="true" stored="true"/>
> >> > <field name="photoDescriptions" type="text" indexed="true"
> >> > stored="true"/>
> >> > <field name="hitCount" type="sint" indexed="true" stored="true"/>
> >> > <field name="voteCount" type="sint" indexed="true" stored="true"/>
> >> > <field name="rating" type="sfloat" indexed="true" stored="true"/>
> >> > <field name="photoTags" type="text" indexed="true" stored="true"/>
> >> > <field name="allText" type="text" indexed="true" stored="false"
> >> > multiValued="true"/>
> >> > </fields>
> >> > <uniqueKey>id</uniqueKey>
> >> >
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >
>
>
>
> --
> --Noble Paul
>
Re: DIH - Document missing required field error
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
oh the names are in CAPS. And in your dataconfig it is in small it
must be something like this
<field column="MYGALLERY_ID" name="id"/>
On Thu, Aug 21, 2008 at 7:33 PM, Todd Breiholz <ta...@gmail.com> wrote:
> On Wed, Aug 20, 2008 at 10:54 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
>> Does the "mygallery_id" column in your database allow nulls? It gets copied
>> to the id field in Solr which is required, hence the error.
>>
>
> Nope. I just confirmed that the mygallery_id column does *not* allow nulls.
> In fact, the debug output from the /dataimport request shows:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">579</int>
> </lst>
> <lst name="initArgs">
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst></lst><str name="command">full-import</str>
> <str name="mode">debug</str>
> <arr name="documents">
> <arr/>
> </arr>
> <lst name="verbose-output">
> <lst name="entity:gallery">
> <lst name="document#1">
> <str name="query">select * from mdp_share_my_search_view
> where mygallery_id = '1301801'</str>
> <str name="time-taken">0:0:0.188</str>
> <str>----------- row #1-------------</str>
> <str name="MYGALLRY_NAME">Princess Party</str>
> <str name="USER_ID">162900018</str>
> <str name="PHOTO_NAMES">Princessparty</str>
> <str name="MYGALLERY_ID">1301801</str>
> <str name="PHOTO_TAGS">Made in Minutes,Photography,Paper and
> cardstockMade in Minutes,SBE Inspired,Embellishing,Cards,Paper and
> cardstock,Paper and cardstockEmbellishing,Cards,Paper and
> cardstock,Paper and cardsto</str>
> <str name="MYGALLRYCHANL_ID">900002</str>
> <str name="HIT_COUNT">java.math.BigDecimal:0</str>
> <str name="MYGALLRYCAT_ID">cat3800005dev</str>
> <null name="RATING"/>
> <str name="PHOTO_DESCS">Saylors 3rd & Sarahs 1st
> birthday were celebrated in style. We had so much fun celebrating our
> princesses birthdays.</str>
> <str name="VOTE_COUNT">java.math.BigDecimal:0</str>
> <str>---------------------------------------------</str>
> </lst>
> <lst name="document#1"/>
> </lst>
> </lst>
> <str name="status">idle</str>
> <str name="importResponse">Configuration Re-loaded sucessfully</str>
> <lst name="statusMessages">
> <str name="Total Requests made to DataSource">1</str>
> <str name="Total Rows Fetched">1</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2008-08-21 08:55:21</str>
> <str name="">Indexing completed. Added/Updated: 0 documents.
> Deleted 0 documents.</str>
> <str name="Committed">2008-08-21 08:55:21</str>
> <str name="Optimized">2008-08-21 08:55:21</str>
> <str name="Time taken ">0:0:0.563</str>
> </lst>
> <str name="WARNING">This response format is experimental. It is
> likely to change in the future.</str>
> </response>
>
>
>
>
>>
>> On Thu, Aug 21, 2008 at 2:12 AM, Todd Breiholz <ta...@gmail.com> wrote:
>>
>> > I am testing out DataImportHandler (Nightly 08-18-2008) in our
>> environment
>> > with a simple schema. One database view with 10 columns mapped to an
>> index
>> > in Solr. When running the full-import command, I get the error below for
>> > each row. I'm sure I'm missing something, but I can't figure it out.
>> >
>> > WARNING: Error creating document : SolrInputDocumnt[{}]
>> > org.apache.solr.common.SolrException: Document [null] missing required
>> > field: id
>> > at
>> >
>> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
>> > at
>> >
>> >
>> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
>> > at
>> > org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
>> > at
>> >
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
>> > at
>> >
>> >
>> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
>> > at
>> >
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
>> > at
>> >
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>> > at
>> >
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>> > at
>> >
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>> > at
>> >
>> >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>> > at
>> >
>> >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>> > at
>> >
>> >
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
>> > at
>> >
>> >
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>> > at
>> >
>> >
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>> > at
>> >
>> >
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>> > at
>> >
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>> > at
>> >
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
>> > at
>> >
>> >
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
>> > at
>> > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
>> > at java.lang.Thread.run(Thread.java:619)
>> >
>> > Here is my data-config.xml:
>> >
>> > <dataConfig>
>> > <dataSource type="JdbcDataSource"
>> > driver="oracle.jdbc.driver.OracleDriver"
>> > url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
>> > <document name="sharemy">
>> > <entity name="gallery" pk="mygallery_id"
>> > query="select * from mdp_share_my_search_view">
>> > <field column="mygallery_id" name="id"/>
>> > <field column="mygallrychanl_id" name="channelId"/>
>> > <field column="mygallrycat_id" name="categoryId"/>
>> > <field column="user_id" name="userId"/>
>> > <field column="mygallry_name" name="galleryName"/>
>> > <field column="photo_names" name="photoNames"/>
>> > <field column="photo_descs" name="photoDescriptions"/>
>> > <field column="hit_count" name="hitCount"/>
>> > <field column="vote_count" name="voteCount"/>
>> > <field column="rating" name="rating"/>
>> > <field column="photo_tags" name="photoTags"/>
>> > </entity>
>> > </document>
>> > </dataConfig>
>> >
>> > and from my schema.xml:
>> >
>> > <fields>
>> > <field name="id" type="string" indexed="true" stored="true"
>> > required="true" />
>> > <field name="channelId" type="string" indexed="true" stored="true"
>> > omitNorms="true"/>
>> > <field name="categoryId" type="string" indexed="true" stored="true"
>> > omitNorms="true"/>
>> > <field name="userId" type="string" indexed="true" stored="true"
>> > omitNorms="true"/>
>> > <field name="galleryName" type="text" indexed="true" stored="true"/>
>> > <field name="photoNames" type="text" indexed="true" stored="true"/>
>> > <field name="photoDescriptions" type="text" indexed="true"
>> > stored="true"/>
>> > <field name="hitCount" type="sint" indexed="true" stored="true"/>
>> > <field name="voteCount" type="sint" indexed="true" stored="true"/>
>> > <field name="rating" type="sfloat" indexed="true" stored="true"/>
>> > <field name="photoTags" type="text" indexed="true" stored="true"/>
>> > <field name="allText" type="text" indexed="true" stored="false"
>> > multiValued="true"/>
>> > </fields>
>> > <uniqueKey>id</uniqueKey>
>> >
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
--
--Noble Paul
Re: DIH - Document missing required field error
Posted by Todd Breiholz <ta...@gmail.com>.
On Wed, Aug 20, 2008 at 10:54 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> Does the "mygallery_id" column in your database allow nulls? It gets copied
> to the id field in Solr which is required, hence the error.
>
Nope. I just confirmed that the mygallery_id column does *not* allow nulls.
In fact, the debug output from the /dataimport request shows:
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">579</int>
</lst>
<lst name="initArgs">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst></lst><str name="command">full-import</str>
<str name="mode">debug</str>
<arr name="documents">
<arr/>
</arr>
<lst name="verbose-output">
<lst name="entity:gallery">
<lst name="document#1">
<str name="query">select * from mdp_share_my_search_view
where mygallery_id = '1301801'</str>
<str name="time-taken">0:0:0.188</str>
<str>----------- row #1-------------</str>
<str name="MYGALLRY_NAME">Princess Party</str>
<str name="USER_ID">162900018</str>
<str name="PHOTO_NAMES">Princessparty</str>
<str name="MYGALLERY_ID">1301801</str>
<str name="PHOTO_TAGS">Made in Minutes,Photography,Paper and
cardstockMade in Minutes,SBE Inspired,Embellishing,Cards,Paper and
cardstock,Paper and cardstockEmbellishing,Cards,Paper and
cardstock,Paper and cardsto</str>
<str name="MYGALLRYCHANL_ID">900002</str>
<str name="HIT_COUNT">java.math.BigDecimal:0</str>
<str name="MYGALLRYCAT_ID">cat3800005dev</str>
<null name="RATING"/>
<str name="PHOTO_DESCS">Saylors 3rd & Sarahs 1st
birthday were celebrated in style. We had so much fun celebrating our
princesses birthdays.</str>
<str name="VOTE_COUNT">java.math.BigDecimal:0</str>
<str>---------------------------------------------</str>
</lst>
<lst name="document#1"/>
</lst>
</lst>
<str name="status">idle</str>
<str name="importResponse">Configuration Re-loaded sucessfully</str>
<lst name="statusMessages">
<str name="Total Requests made to DataSource">1</str>
<str name="Total Rows Fetched">1</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2008-08-21 08:55:21</str>
<str name="">Indexing completed. Added/Updated: 0 documents.
Deleted 0 documents.</str>
<str name="Committed">2008-08-21 08:55:21</str>
<str name="Optimized">2008-08-21 08:55:21</str>
<str name="Time taken ">0:0:0.563</str>
</lst>
<str name="WARNING">This response format is experimental. It is
likely to change in the future.</str>
</response>
>
> On Thu, Aug 21, 2008 at 2:12 AM, Todd Breiholz <ta...@gmail.com> wrote:
>
> > I am testing out DataImportHandler (Nightly 08-18-2008) in our
> environment
> > with a simple schema. One database view with 10 columns mapped to an
> index
> > in Solr. When running the full-import command, I get the error below for
> > each row. I'm sure I'm missing something, but I can't figure it out.
> >
> > WARNING: Error creating document : SolrInputDocumnt[{}]
> > org.apache.solr.common.SolrException: Document [null] missing required
> > field: id
> > at
> >
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
> > at
> >
> >
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
> > at
> > org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
> > at
> >
> >
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
> > at
> >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > at
> >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > at
> >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
> > at
> >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> > at
> >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > at
> >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> > at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
> > at
> >
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> > at
> > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> > at java.lang.Thread.run(Thread.java:619)
> >
> > Here is my data-config.xml:
> >
> > <dataConfig>
> > <dataSource type="JdbcDataSource"
> > driver="oracle.jdbc.driver.OracleDriver"
> > url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
> > <document name="sharemy">
> > <entity name="gallery" pk="mygallery_id"
> > query="select * from mdp_share_my_search_view">
> > <field column="mygallery_id" name="id"/>
> > <field column="mygallrychanl_id" name="channelId"/>
> > <field column="mygallrycat_id" name="categoryId"/>
> > <field column="user_id" name="userId"/>
> > <field column="mygallry_name" name="galleryName"/>
> > <field column="photo_names" name="photoNames"/>
> > <field column="photo_descs" name="photoDescriptions"/>
> > <field column="hit_count" name="hitCount"/>
> > <field column="vote_count" name="voteCount"/>
> > <field column="rating" name="rating"/>
> > <field column="photo_tags" name="photoTags"/>
> > </entity>
> > </document>
> > </dataConfig>
> >
> > and from my schema.xml:
> >
> > <fields>
> > <field name="id" type="string" indexed="true" stored="true"
> > required="true" />
> > <field name="channelId" type="string" indexed="true" stored="true"
> > omitNorms="true"/>
> > <field name="categoryId" type="string" indexed="true" stored="true"
> > omitNorms="true"/>
> > <field name="userId" type="string" indexed="true" stored="true"
> > omitNorms="true"/>
> > <field name="galleryName" type="text" indexed="true" stored="true"/>
> > <field name="photoNames" type="text" indexed="true" stored="true"/>
> > <field name="photoDescriptions" type="text" indexed="true"
> > stored="true"/>
> > <field name="hitCount" type="sint" indexed="true" stored="true"/>
> > <field name="voteCount" type="sint" indexed="true" stored="true"/>
> > <field name="rating" type="sfloat" indexed="true" stored="true"/>
> > <field name="photoTags" type="text" indexed="true" stored="true"/>
> > <field name="allText" type="text" indexed="true" stored="false"
> > multiValued="true"/>
> > </fields>
> > <uniqueKey>id</uniqueKey>
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: DIH - Document missing required field error
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Does the "mygallery_id" column in your database allow nulls? It gets copied
to the id field in Solr which is required, hence the error.
On Thu, Aug 21, 2008 at 2:12 AM, Todd Breiholz <ta...@gmail.com> wrote:
> I am testing out DataImportHandler (Nightly 08-18-2008) in our environment
> with a simple schema. One database view with 10 columns mapped to an index
> in Solr. When running the full-import command, I get the error below for
> each row. I'm sure I'm missing something, but I can't figure it out.
>
> WARNING: Error creating document : SolrInputDocumnt[{}]
> org.apache.solr.common.SolrException: Document [null] missing required
> field: id
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
> at
>
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:58)
> at
> org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:69)
> at
>
> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:288)
> at
>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:317)
> at
>
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:176)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:134)
> at
>
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:332)
> at
>
> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:384)
> at
>
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:190)
> at
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1188)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
> at
>
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
>
> Here is my data-config.xml:
>
> <dataConfig>
> <dataSource type="JdbcDataSource"
> driver="oracle.jdbc.driver.OracleDriver"
> url="jdbc:oracle:oci:@orcl" user="..." password="..."/>
> <document name="sharemy">
> <entity name="gallery" pk="mygallery_id"
> query="select * from mdp_share_my_search_view">
> <field column="mygallery_id" name="id"/>
> <field column="mygallrychanl_id" name="channelId"/>
> <field column="mygallrycat_id" name="categoryId"/>
> <field column="user_id" name="userId"/>
> <field column="mygallry_name" name="galleryName"/>
> <field column="photo_names" name="photoNames"/>
> <field column="photo_descs" name="photoDescriptions"/>
> <field column="hit_count" name="hitCount"/>
> <field column="vote_count" name="voteCount"/>
> <field column="rating" name="rating"/>
> <field column="photo_tags" name="photoTags"/>
> </entity>
> </document>
> </dataConfig>
>
> and from my schema.xml:
>
> <fields>
> <field name="id" type="string" indexed="true" stored="true"
> required="true" />
> <field name="channelId" type="string" indexed="true" stored="true"
> omitNorms="true"/>
> <field name="categoryId" type="string" indexed="true" stored="true"
> omitNorms="true"/>
> <field name="userId" type="string" indexed="true" stored="true"
> omitNorms="true"/>
> <field name="galleryName" type="text" indexed="true" stored="true"/>
> <field name="photoNames" type="text" indexed="true" stored="true"/>
> <field name="photoDescriptions" type="text" indexed="true"
> stored="true"/>
> <field name="hitCount" type="sint" indexed="true" stored="true"/>
> <field name="voteCount" type="sint" indexed="true" stored="true"/>
> <field name="rating" type="sfloat" indexed="true" stored="true"/>
> <field name="photoTags" type="text" indexed="true" stored="true"/>
> <field name="allText" type="text" indexed="true" stored="false"
> multiValued="true"/>
> </fields>
> <uniqueKey>id</uniqueKey>
>
--
Regards,
Shalin Shekhar Mangar.