session support for nutch

[ma...@provinzial.com]

Dear Nutchers,

I am in development of session support for nutch. A session is formally an
extension to resource location, but a session also carries some
authentificated information. Session configurations are inheritable,
to ease maintenance.

The use of sessions makes sensible to expand nutch-conf to be session
specific. for e.g. using separate URLFilter or Metadata for each defined
session.

I have a problem with some components, especially with plugins. At this
time plugins get configured with the configuration in use at the plugin
load time.

Pseudo Code:

 for (Iterator i = SessionConf.iterator(); i.hasNext();)  {
      String sessionId = (String) i.next();
      SessionConf.Session session =  SessionConf.get(sessionId);
      NutchConf.OpenConf(session.getNutchConf());
      try {
         ...
         Plugin.Call();
         ...
      } finally {
        NutchConf.CloseConf();
      }
}

OpenConf/CloseConf each change thread local the configuration set
NutchConf.get() will return in future. This is why the plugin is configured
to the first configuration, when it’s called the second time.

Especially: A plugins configuration data is likely to change between to
calls.

I think I am in need of a separate set of plugins for every session.

What do you think, would be the correct place to implement this into the
plugin system?

Marcel Schnippe