You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Pattabiraman, Meenakshisundaram" <Pa...@aig.com> on 2015/06/18 03:33:29 UTC
XPathentity processor on CLOB field
My requirement is to read the XML from a CLOB field and parse it to get the entity.
The data config is as shown below. I am trying to map two fields 'event' and 'policyNumber' for the entity 'catreport'.
<dataSource name="mbdev" driver="oracle.jdbc.driver.OracleDriver" url="jdbc:oracle:thin:@localhost:1521:orcl" user="xyz" password="xyz"/>
<document name="insight">
<entity name="input" query="select * from test" logLevel="debug" datasource="mbdev" transformer="ClobTransformer, script:toDate">
<field column="LOAD_DATE" name="load_date" />
<field column="RESPONSE_XML" name="RESPONSE_XML" clob="true" />
<dataSource name="xmldata" type="FieldReaderDataSource"/>
<entity name="catReport" dataSource="xmldata" dataField="input.RESPONSE_XML" processor="XPathEntityProcessor" forEach="/*:DecisionServiceRs" rootEntity="true" logLevel="debug">
<field column="event" xpath="/dec:DecisionServiceRs/@event"/>
I am getting this error
....
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: null Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:321)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:278)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:53)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.initQuery(XPathEntityProcessor.java:283)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.fetchNextRow(XPathEntityProcessor.java:224)
I see that the Clob is getting converted to String correctly and the log has this entry where xml is printed
Exception while processing: input document : SolrInputDocument(fields: [RESPONSE_XML=<dec:Deci....
I do not know why the error is thrown at Jdbc when the Clob is converted to string and passed to the FieldReader and do not know how to make this work.
Thanks
Pattabi
RE: XPathentity processor on CLOB field
Posted by "Pattabiraman, Meenakshisundaram" <Pa...@aig.com>.
I got this working - the errors were due to a mistake in letter case - was using 'datasource' instead of 'dataSource' in the entity that was using XpathEntityProcessor. Hence this was being ignored and was inheriting the JDBC Datasource of the parent entity.
I am pasting the complete data-config for anyone encountering the same problem.
<dataConfig>
<dataSource name="xmldata" type="FieldReaderDataSource"/>
<dataSource name="mbdev" driver="oracle.jdbc.driver.OracleDriver" url="jdbc:oracle:thin:@localhost:1521:orcl" user="orcl" password="orcl"/>
<document name="insight">
<entity name="input" query="select * from test" logLevel="debug" dataSource="mbdev" transformer="ClobTransformer" onError="skip">
<field column="LOAD_DATE" name="load_date" />
<field column="RESPONSE_XML" name="RESPONSE_XML" clob="true" />
<field column="id" name="id"/>
<entity name="catReport" dataSource="xmldata" dataField="input.RESPONSE_XML" processor="XPathEntityProcessor" forEach="/DecisionServiceRs" rootEntity="true" logLevel="debug">
<field column="event" xpath="/DecisionServiceRs/@event"/>
<field column="policyNumber" xpath="/DecisionServiceRs/@policyNumber"/>
</entity>
</entity>
</document>
</dataConfig>
-----Original Message-----
From: Pattabiraman, Meenakshisundaram [mailto:Pattabiraman.Meenakshisundaram@aig.com]
Sent: Thursday, June 18, 2015 7:51 AM
To: solr-user@lucene.apache.org
Subject: RE: XPathentity processor on CLOB field
This is the error cause reported. I also see that it has been reported earlier (http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201103.mbox/%3CD0F0D26C-3AC0-4982-9E2B-09DC96937193@535consulting.com%3E) but could not find a solution.
I am nesting the FieldReaderDataSource within the Entity definition that has a CLOB field. With this it fails only after transforming the clob.
If I do not nest, I get this error when the FieldReaderDataSource is initialized thus failing even before the SQL is executed.
Either case, the error is happening at the same place.
Caused by: java.sql.SQLException: SQL statement to execute cannot be empty or null
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:229)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:403)
at oracle.jdbc.driver.OracleSql.initialize(OracleSql.java:110)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1761)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1739)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:298)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:314)
... 14 more
Pattabi Meenakshisundaram
-----Original Message-----
From: Pattabiraman, Meenakshisundaram [mailto:Pattabiraman.Meenakshisundaram@aig.com]
Sent: Wednesday, June 17, 2015 9:33 PM
To: 'solr-user@lucene.apache.org'
Subject: XPathentity processor on CLOB field
My requirement is to read the XML from a CLOB field and parse it to get the entity.
The data config is as shown below. I am trying to map two fields 'event' and 'policyNumber' for the entity 'catreport'.
<dataSource name="mbdev" driver="oracle.jdbc.driver.OracleDriver" url="jdbc:oracle:thin:@localhost:1521:orcl" user="xyz" password="xyz"/> <document name="insight">
<entity name="input" query="select * from test" logLevel="debug" datasource="mbdev" transformer="ClobTransformer, script:toDate">
<field column="LOAD_DATE" name="load_date" />
<field column="RESPONSE_XML" name="RESPONSE_XML" clob="true" /> <dataSource name="xmldata" type="FieldReaderDataSource"/>
<entity name="catReport" dataSource="xmldata" dataField="input.RESPONSE_XML" processor="XPathEntityProcessor" forEach="/*:DecisionServiceRs" rootEntity="true" logLevel="debug">
<field column="event" xpath="/dec:DecisionServiceRs/@event"/>
I am getting this error
....
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: null Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:321)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:278)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:53)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.initQuery(XPathEntityProcessor.java:283)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.fetchNextRow(XPathEntityProcessor.java:224)
I see that the Clob is getting converted to String correctly and the log has this entry where xml is printed Exception while processing: input document : SolrInputDocument(fields: [RESPONSE_XML=<dec:Deci....
I do not know why the error is thrown at Jdbc when the Clob is converted to string and passed to the FieldReader and do not know how to make this work.
Thanks
Pattabi
RE: XPathentity processor on CLOB field
Posted by "Pattabiraman, Meenakshisundaram" <Pa...@aig.com>.
This is the error cause reported. I also see that it has been reported earlier (http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201103.mbox/%3CD0F0D26C-3AC0-4982-9E2B-09DC96937193@535consulting.com%3E) but could not find a solution.
I am nesting the FieldReaderDataSource within the Entity definition that has a CLOB field. With this it fails only after transforming the clob.
If I do not nest, I get this error when the FieldReaderDataSource is initialized thus failing even before the SQL is executed.
Either case, the error is happening at the same place.
Caused by: java.sql.SQLException: SQL statement to execute cannot be empty or null
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:229)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:403)
at oracle.jdbc.driver.OracleSql.initialize(OracleSql.java:110)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1761)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1739)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:298)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:314)
... 14 more
Pattabi Meenakshisundaram
-----Original Message-----
From: Pattabiraman, Meenakshisundaram [mailto:Pattabiraman.Meenakshisundaram@aig.com]
Sent: Wednesday, June 17, 2015 9:33 PM
To: 'solr-user@lucene.apache.org'
Subject: XPathentity processor on CLOB field
My requirement is to read the XML from a CLOB field and parse it to get the entity.
The data config is as shown below. I am trying to map two fields 'event' and 'policyNumber' for the entity 'catreport'.
<dataSource name="mbdev" driver="oracle.jdbc.driver.OracleDriver" url="jdbc:oracle:thin:@localhost:1521:orcl" user="xyz" password="xyz"/> <document name="insight">
<entity name="input" query="select * from test" logLevel="debug" datasource="mbdev" transformer="ClobTransformer, script:toDate">
<field column="LOAD_DATE" name="load_date" />
<field column="RESPONSE_XML" name="RESPONSE_XML" clob="true" /> <dataSource name="xmldata" type="FieldReaderDataSource"/>
<entity name="catReport" dataSource="xmldata" dataField="input.RESPONSE_XML" processor="XPathEntityProcessor" forEach="/*:DecisionServiceRs" rootEntity="true" logLevel="debug">
<field column="event" xpath="/dec:DecisionServiceRs/@event"/>
I am getting this error
....
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: null Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:70)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:321)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:278)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:53)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.initQuery(XPathEntityProcessor.java:283)
at org.apache.solr.handler.dataimport.XPathEntityProcessor.fetchNextRow(XPathEntityProcessor.java:224)
I see that the Clob is getting converted to String correctly and the log has this entry where xml is printed Exception while processing: input document : SolrInputDocument(fields: [RESPONSE_XML=<dec:Deci....
I do not know why the error is thrown at Jdbc when the Clob is converted to string and passed to the FieldReader and do not know how to make this work.
Thanks
Pattabi