You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Enrico Triolo <en...@gmail.com> on 2006/02/09 13:43:26 UTC

Changing property value at runtime

Is there a way of changing at runtime properties values defined in a
configuration resource?

I mean, let's say I created a configuration resource (my-conf.xml) with this
content:

<configuration>
        <property>
                <name>wsId</name>
                <value>0</value>
        </property>
</configuration>

I then developed a simple class (cut and pasting from Crawl.java main
method) with a method like this:

    public void addUrl(String url, int wsId) throws Exception {

        Configuration conf = NutchConfiguration.create();
        JobConf job = new JobConf(conf);
        job.addDefaultResource("my-conf.xml");

        //This doesn't work!
        //job.set("wsId", "" + wsId);

        //From here on, it's just cut & paste from Crawl.main() with some
arrangement!

        // initialize crawlDb
        new Injector(job).inject(crawlDbDir, new File(wsDirName + "urls/"));

        File segment = new Generator(this.job).generate(crawlDbDir,
segmentsDir, -1, topN, System.currentTimeMillis());
        new Fetcher(this.job).fetch(segment, threads, Fetcher.isParsing(
this.job));  // fetch it
        if (!Fetcher.isParsing(this.job))
           new ParseSegment(this.job).parse(segment);    // parse it, if
needed
        new CrawlDb(this.job).update(crawlDbDir, segment); // update crawldb

        new LinkDb(this.job).invert(linkDbDir, segmentsDir); // invert links

        // index, dedup & merge
        new Indexer(this.job).index(indexesDir, crawlDbDir, linkDbDir,
this.nfs.listFiles(segmentsDir));
        new DeleteDuplicates(this.job).dedup(new File[] { indexesDir });
        new IndexMerger(this.nfs, this.nfs.listFiles(indexesDir), indexDir,
tmpDir, this.job).merge();
    }

This way I managed to add "wsId" property, and can retrieve it from my
plugins. But wsId is a dynamic property and I need to set it
programmatically based on the homonymous parameter passed to the addUrl
method.
As you can see in the code, I tried to manually set the property, but if I
print its value in the plugin I always get "0".

Is there a way of handling such a situation? Also, I'm I writing to the
correct mailing list or should I post my question to the devel ml?

Thanks in advance,
Enrico