You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Enrico Triolo <en...@gmail.com> on 2006/02/09 13:43:26 UTC
Changing property value at runtime
Is there a way of changing at runtime properties values defined in a
configuration resource?
I mean, let's say I created a configuration resource (my-conf.xml) with this
content:
<configuration>
<property>
<name>wsId</name>
<value>0</value>
</property>
</configuration>
I then developed a simple class (cut and pasting from Crawl.java main
method) with a method like this:
public void addUrl(String url, int wsId) throws Exception {
Configuration conf = NutchConfiguration.create();
JobConf job = new JobConf(conf);
job.addDefaultResource("my-conf.xml");
//This doesn't work!
//job.set("wsId", "" + wsId);
//From here on, it's just cut & paste from Crawl.main() with some
arrangement!
// initialize crawlDb
new Injector(job).inject(crawlDbDir, new File(wsDirName + "urls/"));
File segment = new Generator(this.job).generate(crawlDbDir,
segmentsDir, -1, topN, System.currentTimeMillis());
new Fetcher(this.job).fetch(segment, threads, Fetcher.isParsing(
this.job)); // fetch it
if (!Fetcher.isParsing(this.job))
new ParseSegment(this.job).parse(segment); // parse it, if
needed
new CrawlDb(this.job).update(crawlDbDir, segment); // update crawldb
new LinkDb(this.job).invert(linkDbDir, segmentsDir); // invert links
// index, dedup & merge
new Indexer(this.job).index(indexesDir, crawlDbDir, linkDbDir,
this.nfs.listFiles(segmentsDir));
new DeleteDuplicates(this.job).dedup(new File[] { indexesDir });
new IndexMerger(this.nfs, this.nfs.listFiles(indexesDir), indexDir,
tmpDir, this.job).merge();
}
This way I managed to add "wsId" property, and can retrieve it from my
plugins. But wsId is a dynamic property and I need to set it
programmatically based on the homonymous parameter passed to the addUrl
method.
As you can see in the code, I tried to manually set the property, but if I
print its value in the plugin I always get "0".
Is there a way of handling such a situation? Also, I'm I writing to the
correct mailing list or should I post my question to the devel ml?
Thanks in advance,
Enrico