Loading custom update request handler on startup

[Jay Hill <ja...@gmail.com>]

I'm writing a custom update request handler that will poll a "hot"
directory for Solr xml files and index anything it finds there. The custom
class implements Runnable, and when the run method is called the loop
starts to do the polling. How can I tell Solr to load this class on startup
to fire off the run() method?

Thanks,
-Jay

Re: Loading custom update request handler on startup

[Erik Hatcher <er...@gmail.com>]

Couldn't you do this in the init() method?   You might have to kick off your background thread, but lazy initialize stuff when it fires to initialize other things that require Solr be fully ready.

Note that DataImportHandler can index Solr XML files with very little configuration, and cronning firing of that might be an option worth considering (though I'm not sure if it does incremental and picks up where it left off?)

	Erik


On Jul 9, 2012, at 18:49 , Jay Hill wrote:

> I may have found a good solution. I implemented my own SolrEventListener:
> 
> public class DynamicIndexerEventListener
> implementsorg.apache.solr.core.SolrEventListener{
> 
> ...
> 
> and then called it with a "firstSearcher" element in solrconfig.xml:
> 
>    <listener event="firstSearcher"
> class="com.bestbuy.search.foundation.solr.DynamicIndexerEventListener" />
> 
> Then in the newSearcher() method I startup up the thread for my polling
> UpdateRequestHandler.
> 
> This seems to work, but if anyone has a better (or more tested) approach
> please let us know.
> 
> 
> -Jay
> 
> On Mon, Jul 9, 2012 at 2:33 PM, Jay Hill <ja...@gmail.com> wrote:
> 
>> I'm writing a custom update request handler that will poll a "hot"
>> directory for Solr xml files and index anything it finds there. The custom
>> class implements Runnable, and when the run method is called the loop
>> starts to do the polling. How can I tell Solr to load this class on startup
>> to fire off the run() method?
>> 
>> Thanks,
>> -Jay
>> 

Re: Loading custom update request handler on startup

[Michel Dion <di...@gmail.com>]

Another approach would be to use an external application executed by a cron
or some scheduler that would post the file to solr using the class.

 org.apache.solr.util.SimplePostTool

SimplePostTool postTool = new SimplePostTool(new URL(SOLR_URL));

for (File file : outputDir.listFiles(/* smoe filtering */) ){
  try {
      postTool.postFile(file, System.out, "xml");
      //move or delete the posted files
      boolean success = file.renameTo(new File(postedDir, file.getName()));
      if (!success) {
       LOGGER.info("File was not successfully moved :"+file.getName());
      }
  }
  catch (Exception e) {
      LOGGER.error("Error during file post :" + file.getCanonicalPath(), e);
      //behavior to take if one file couldn't be posted, we stop or continue
      if (failfast) {
          return;
      }
  }
}
 // solr commit
 postTool.commit();

Attention must be paid on the coordination between the schedule and the
"hot directory" to avoid posting files that wouldn't be entirely written:
to avoid posting  uncompleted files.

Regards,
Michel


On Mon, Jul 9, 2012 at 6:49 PM, Jay Hill <ja...@gmail.com> wrote:

> I may have found a good solution. I implemented my own SolrEventListener:
>
> public class DynamicIndexerEventListener
> implementsorg.apache.solr.core.SolrEventListener{
>
> ...
>
> and then called it with a "firstSearcher" element in solrconfig.xml:
>
>     <listener event="firstSearcher"
> class="com.bestbuy.search.foundation.solr.DynamicIndexerEventListener" />
>
> Then in the newSearcher() method I startup up the thread for my polling
> UpdateRequestHandler.
>
> This seems to work, but if anyone has a better (or more tested) approach
> please let us know.
>
>
> -Jay
>
> On Mon, Jul 9, 2012 at 2:33 PM, Jay Hill <ja...@gmail.com> wrote:
>
> > I'm writing a custom update request handler that will poll a "hot"
> > directory for Solr xml files and index anything it finds there. The
> custom
> > class implements Runnable, and when the run method is called the loop
> > starts to do the polling. How can I tell Solr to load this class on
> startup
> > to fire off the run() method?
> >
> > Thanks,
> > -Jay
> >
>

Re: Loading custom update request handler on startup

[Jay Hill <ja...@gmail.com>]

I may have found a good solution. I implemented my own SolrEventListener:

public class DynamicIndexerEventListener
implementsorg.apache.solr.core.SolrEventListener{

...

and then called it with a "firstSearcher" element in solrconfig.xml:

    <listener event="firstSearcher"
class="com.bestbuy.search.foundation.solr.DynamicIndexerEventListener" />

Then in the newSearcher() method I startup up the thread for my polling
UpdateRequestHandler.

This seems to work, but if anyone has a better (or more tested) approach
please let us know.


-Jay

On Mon, Jul 9, 2012 at 2:33 PM, Jay Hill <ja...@gmail.com> wrote:

> I'm writing a custom update request handler that will poll a "hot"
> directory for Solr xml files and index anything it finds there. The custom
> class implements Runnable, and when the run method is called the loop
> starts to do the polling. How can I tell Solr to load this class on startup
> to fire off the run() method?
>
> Thanks,
> -Jay
>