You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by devang pandey <de...@gmail.com> on 2013/07/26 09:14:40 UTC

crawl time details of a particular domain

Hello , I am working on nutch 1.4 to crawl certain domains . Now after
successful crawling I want to get start crawl time and end crawl time
corresponding to each and every domain . How can I achieve this . Please
guide me .

Re: crawl time details of a particular domain

Posted by Lewis John Mcgibbney <le...@gmail.com>.
You can look in to the Fetcher#fetch(Path segment, int threads) method [0],
where

1284 SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
1285 long start = System.currentTimeMillis();  1286 if
(LOG.isInfoEnabled()) {  1287 LOG.info("Fetcher: starting at " +
sdf.format(start));...
1342 long end = System.currentTimeMillis();  1343 LOG.info("Fetcher:
finished at " + sdf.format(end) + ", elapsed: " +
TimingUtil.elapsedTime(start, end));
[0]
http://svn.apache.org/viewvc/nutch/trunk/src/java/org/apache/nutch/fetcher/Fetcher.java?view=markup



On Mon, Jul 29, 2013 at 2:37 AM, devang pandey <de...@gmail.com>wrote:

> @lewis ... Thanx for replying . Thing is using readdb I can read my crawldb
> but it shows only fetcher time . How can i find Fetcher start and end time
> as suggested by you..
>
>
> On Sat, Jul 27, 2013 at 6:05 AM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > by looking at the fetcher start time and finish time Sir.
> > There should be mothods in TimingUtil to help you.
> > hth
> >
> > On Friday, July 26, 2013, devang pandey <de...@gmail.com>
> wrote:
> > > Hello , I am working on nutch 1.4 to crawl certain domains . Now after
> > > successful crawling I want to get start crawl time and end crawl time
> > > corresponding to each and every domain . How can I achieve this .
> Please
> > > guide me .
> > >
> >
> > --
> > *Lewis*
> >
>



-- 
*Lewis*

Re: crawl time details of a particular domain

Posted by devang pandey <de...@gmail.com>.
@lewis ... Thanx for replying . Thing is using readdb I can read my crawldb
but it shows only fetcher time . How can i find Fetcher start and end time
as suggested by you..


On Sat, Jul 27, 2013 at 6:05 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> by looking at the fetcher start time and finish time Sir.
> There should be mothods in TimingUtil to help you.
> hth
>
> On Friday, July 26, 2013, devang pandey <de...@gmail.com> wrote:
> > Hello , I am working on nutch 1.4 to crawl certain domains . Now after
> > successful crawling I want to get start crawl time and end crawl time
> > corresponding to each and every domain . How can I achieve this . Please
> > guide me .
> >
>
> --
> *Lewis*
>

Re: crawl time details of a particular domain

Posted by Lewis John Mcgibbney <le...@gmail.com>.
by looking at the fetcher start time and finish time Sir.
There should be mothods in TimingUtil to help you.
hth

On Friday, July 26, 2013, devang pandey <de...@gmail.com> wrote:
> Hello , I am working on nutch 1.4 to crawl certain domains . Now after
> successful crawling I want to get start crawl time and end crawl time
> corresponding to each and every domain . How can I achieve this . Please
> guide me .
>

-- 
*Lewis*