DataImportHandlerException when cache key is null in SOLR 1.4.1

[Paul Dhaliwal <su...@gmail.com>]

Hello,

I am using CachedSqlEntityProcessor for one of my entities and it working
fine when I am loading the cache as keys are requested, however it doesn't
work when I try to pre load the cache. My configuration for "loading the
cache as keys are requested" is:

<entity dataSource="ds"  name="app" pk="app_id"
processor="CachedSqlEntityProcessor"
                    query="select * from m_app where
app_id='${m_app_root.app_id}'">

When I change it to the following config to "pre load the cache"

<entity dataSource="ds"  name="app" pk="app_id"
processor="CachedSqlEntityProcessor"
                    query="select * from m_app"
where="app_id=m_app_root.app_id">

I start getting the following exceptions:
SEVERE: Exception while processing: m_app_root document :
SolrInputDocument[{......}]
org.apache.solr.handler.dataimport.DataImportHandlerException: The cache
lookup value :m_app_root.app_id is resolved to be null in the entity :app
Processing Document # 81671


I looked at the code and this is what is there:

EntityProcessorBase.getIdCacheData (line 178)

...
 if (key == null) {
      throw new DataImportHandlerException(DataImportHandlerException.WARN,
              "The cache lookup value : " + cacheVariableName + " is
resolved to be null in the entity :" +
                      context.getEntityAttribute("name"));

    }
...

I think it should be logged, but adding an exception is an overkill because
the whole SolrInputDocument is rejected.
I don't always expect a sub entity to be present because fields being
populated are nullable. Furthermore, my lookup key for sub entity can be
null in the root entity, because its not required to be present in the
schema. I am sure lot of other people have the same case. Also it works fine
in the non pre load version of CachedSqlEntityProcessor.

Here is the jira link:

https://issues.apache.org/jira/browse/SOLR-884

and diff link:
http://svn.apache.org/viewvc/lucene/solr/trunk/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/EntityProcessorBase.java?p2=%2Flucene%2Fsolr%2Ftrunk%2Fcontrib%2Fdataimporthandler%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fsolr%2Fhandler%2Fdataimport%2FEntityProcessorBase.java&p1=%2Flucene%2Fsolr%2Ftrunk%2Fcontrib%2Fdataimporthandler%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fsolr%2Fhandler%2Fdataimport%2FEntityProcessorBase.java&r1=725684&r2=725683&view=diff&pathrev=725684

Any ideas?

Thanks,
Paul

Re: DataImportHandlerException when cache key is null in SOLR 1.4.1

[Alexey Serba <as...@gmail.com>]

> I think it should be logged, but adding an exception is an overkill because
> the whole SolrInputDocument is rejected.
Makes sense. Could you please file a ticket and attach a patch to it?

It would be great to have single (not one per doc/row) warning
message, otherwise with a big result set with nulls it would pollute
the logs.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org