You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Matt Parker (Commented) (JIRA)" <ji...@apache.org> on 2011/11/25 03:12:40 UTC
[jira] [Commented] (SOLR-2914)
UpdateRequestProcessor.proccessCommit Method not Called When Running DIH in
Debug Mode
[ https://issues.apache.org/jira/browse/SOLR-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156960#comment-13156960 ]
Matt Parker commented on SOLR-2914:
-----------------------------------
I created a class that extends UpdateRequestProcessor, which logs calls to each method. In the console output, you'll see that only the processAdd methods are called.
I've attached the SOLR configuration that can be used to replicate the issue. I'm assuming that once Lucene commits the files, the UpdateRequestProcessor.processCommit method should be called too.
*UpdateRequestProcessorTest.class*
package org.apache.solr.handler.dataimport;
import java.io.IOException;
import org.apache.solr.update.AddUpdateCommand;
import org.apache.solr.update.CommitUpdateCommand;
import org.apache.solr.update.DeleteUpdateCommand;
import org.apache.solr.update.MergeIndexesCommand;
import org.apache.solr.update.RollbackUpdateCommand;
import org.apache.solr.update.processor.UpdateRequestProcessor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Class used to show the interaction between the
* DIH process and the UpdateRequestProcessor chain.
*
*/
public class UpdateRequestProcessorTest extends UpdateRequestProcessor
{
protected static final Logger log = LoggerFactory.getLogger(UpdateRequestProcessorTest.class);
public UpdateRequestProcessorTest(UpdateRequestProcessor next) {
super(next);
}
public void finish() throws IOException {
log.info("finish called...");
super.finish();
}
public void processAdd(AddUpdateCommand cmd) throws IOException {
log.info("processAdd called...");
super.processAdd(cmd);
}
public void processDelete(DeleteUpdateCommand cmd) throws IOException {
log.info("processDelete called...");
super.processDelete(cmd);
}
public void processMergeIndexes(MergeIndexesCommand cmd) throws IOException {
log.info("processMergeIndexes called...");
super.processMergeIndexes(cmd);
}
public void processRollback(RollbackUpdateCommand cmd) throws IOException {
log.info("processRollback called...");
super.processRollback(cmd);
}
public void processCommit(CommitUpdateCommand cmd) throws IOException {
log.info("processCommit called...");
super.processCommit(cmd);
}
}
*UpdateRequestProcessorTestFactory.class*
package org.apache.solr.handler.dataimport;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.request.SolrQueryResponse;
import org.apache.solr.update.processor.UpdateRequestProcessor;
import org.apache.solr.update.processor.UpdateRequestProcessorFactory;
/**
* Factory used to instantiate the UpdateRequestProcessorTest class
* in the UpdateRequest processor chain.
*
*/
public class UpdateRequestProcessorTestFactory extends UpdateRequestProcessorFactory {
public UpdateRequestProcessor getInstance( SolrQueryRequest req, SolrQueryResponse rsp, UpdateRequestProcessor next ){
return new UpdateRequestProcessorTest(next);
}
}
*Console Output*
Nov 24, 2011 8:50:39 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null params={start=0&event=firstSearcher&q=solr+rocks&rows=10} hits=0 status=0 QTime=74
Nov 24, 2011 8:50:39 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null params={event=firstSearcher&q=static+firstSearcher+warming+query+from+solrconfig.xml} hits=85 status=0 QTime=27
Nov 24, 2011 8:50:39 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Nov 24, 2011 8:50:39 PM org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener newSearcher
INFO: Loading spell index for spellchecker: default
Nov 24, 2011 8:50:39 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@6e4eeaaf main
Nov 24, 2011 8:51:32 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={command=status&qt=/solr-2914-test-case} status=0 QTime=7
Nov 24, 2011 8:51:32 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={command=show-config&qt=/solr-2914-test-case} status=0 QTime=1
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DataImportHandler processConfiguration
INFO: Processing configuration from solrconfig.xml: {config=./file-data-source-config.xml,update.processor=dih_test}
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DataImporter loadDataConfig
INFO: Data Configuration loaded successfully
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DataImporter verifyWithSchema
INFO: id is a required field in SolrSchema . But not found in DataConfig
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DataImporter doFullImport
INFO: Starting Full Import
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.SolrWriter readIndexerProperties
INFO: Read solr-2914-test-case.properties
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.LogTransformer transformRow
INFO: Processing C:\Projects\solr-test-cases\SOLR-2914\solr\sample-data\artcle1.txt
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.UpdateRequestProcessorTest processAdd
INFO: processAdd called...
Nov 24, 2011 8:51:41 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=C:\Projects\apache-solr-1.4.1\example\solr\data\index,segFN=segments_g,version=1322170573144,generation=16,filenames=[_l.fnm, _m.frq, _m.prx, _l.frq, _n.tis, _m.tis, _l.tis, _n.frq, _n.tii, _m.tii, _l.tii,
_m.fnm, _n.fnm, _l.nrm, _n.nrm, _l.prx, _n.prx, _n.fdt, _n.fdx, _m.nrm, segments_g, _m.fdt, _l.fdx, _m.fdx, _l.fdt]
Nov 24, 2011 8:51:41 PM org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1322170573144
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.LogTransformer transformRow
INFO: Processing C:\Projects\solr-test-cases\SOLR-2914\solr\sample-data\article2.txt
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.UpdateRequestProcessorTest *processAdd*
INFO: processAdd called...
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.LogTransformer transformRow
INFO: Processing C:\Projects\solr-test-cases\SOLR-2914\solr\sample-data\article3.txt
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.UpdateRequestProcessorTest *processAdd*
INFO: processAdd called...
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.LogTransformer transformRow
INFO: Processing C:\Projects\solr-test-cases\SOLR-2914\solr\sample-data\article4.txt
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.UpdateRequestProcessorTest *processAdd*
INFO: processAdd called...
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.LogTransformer transformRow
INFO: Processing C:\Projects\solr-test-cases\SOLR-2914\solr\sample-data\article5.txt
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.UpdateRequestProcessorTest *processAdd*
INFO: processAdd called...
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DocBuilder finish
INFO: Import completed successfully
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.SolrWriter readIndexerProperties
INFO: Read solr-2914-test-case.properties
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.SolrWriter persist
INFO: Wrote last indexed time to solr-2914-test-case.properties
Nov 24, 2011 8:51:41 PM org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:0.270
Nov 24, 2011 8:51:41 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={commit=on&start=0&dataConfig=<?xml+version%3D'1.0'?>%0d%0a+<dataConfig>%0d%0a++++<dataSource+type%3D"FileDataSource"+/>%0d%0a++++<document>%0d%0a++++++++<entity+name%3D"f"+processor
%3D"FileListEntityProcessor"+baseDir%3D"C:/Projects/solr-test-cases/SOLR-2914/solr/sample-data"+fileName%3D".*txt"+rootEntity%3D"false"+dataSource%3D"null">%0d%0a%09%09++++<entity+processor%3D"PlainTextEntityProcessor"+name%
3D"x"+url%3D"${f.fileAbsolutePath}"+dataSource%3D"f"+transformer%3D"LogTransformer"+logTemplate%3D"++++++Processing+${f.fileAbsolutePath}"+logLevel%3D"info">%0d%0a++++++++++++++++++<!--+copies+the+text+to+a+field+called+'tex
t'+in+Solr-->%0d%0a++++++++++++++++<field+column%3D"plainText"+name%3D"text"/>%0d%0a++++++++++++</entity>%0d%0a++++++++</entity>%0d%0a++++</document>%0d%0a</dataConfig>&verbose=on&command=full-import&debug=on&qt=/solr-2914-t
est-case&rows=10} status=0 QTime=291
> UpdateRequestProcessor.proccessCommit Method not Called When Running DIH in Debug Mode
> --------------------------------------------------------------------------------------
>
> Key: SOLR-2914
> URL: https://issues.apache.org/jira/browse/SOLR-2914
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1
> Environment: SOLR 1.4.1, JDK 1.6.0_23
> Reporter: Matt Parker
> Priority: Minor
>
> Users can test their DIH import configuration files using the dataimport.jsp Admin page. If you check the commit box, and then run the DIH in debug mode, Lucene will commit the records to the index, but the UpdateRequestProcess.processCommit methods are not called. If you run the same DIH as a full import, the DIH will commit the documents, and the processCommit methods are called.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org