You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Savannah Beckett <sa...@yahoo.com> on 2010/07/20 06:41:41 UTC
mysql
Hi,
I use DriverManager.getConnection to connect to my mysql db and do query in my
plugin that extend indexfilter. It seems that indexfilter is run for every url
being indexed. It means that I have to open and close connection to mysql db
each time a url is being indexed, it is unefficient. Is there a way to open and
close connection to mysql db only once for all urls, not for each url?
Thanks.
RE: mysql
Posted by Ar...@csiro.au.
Yes, it is an extension of Nutch. You could use its index filter code as an example. It does exactly what you want to do. I will send you the source off the list.
>-----Original Message-----
>From: Savannah Beckett [mailto:savannah_beckett30@yahoo.com]
>Sent: Tuesday, July 20, 2010 2:50 PM
>To: user@nutch.apache.org
>Subject: Re: mysql
>
>I don't understand, it is some kind of a extension of nutch which seems
>to have
>nothing to do with my question. Even if it does, it seems to be a
>overkill...
>
>
>
>
>________________________________
>From: "Arkadi.Kosmynin@csiro.au" <Ar...@csiro.au>
>To: user@nutch.apache.org
>Sent: Mon, July 19, 2010 9:46:03 PM
>Subject: RE: mysql
>
>Hi Savannah,
>
>>-----Original Message-----
>>From: Savannah Beckett [mailto:savannah_beckett30@yahoo.com]
>>Sent: Tuesday, July 20, 2010 2:42 PM
>>To: user@nutch.apache.org
>>Subject: mysql
>>
>>Hi,
>> I use DriverManager.getConnection to connect to my mysql db and do
>>query in my
>>plugin that extend indexfilter. It seems that indexfilter is run for
>>every url
>>being indexed. It means that I have to open and close connection to
>>mysql db
>>each time a url is being indexed, it is unefficient. Is there a way to
>>open and
>>close connection to mysql db only once for all urls, not for each url?
>
>Yes, there is. See Arch index filter:
>
>http://www.atnf.csiro.au/computing/software/arch/
>
>Regards,
>
>Arkadi
>
>>
>>Thanks.
>>
>>
>>
>
>
>
>
Re: mysql
Posted by Savannah Beckett <sa...@yahoo.com>.
I don't understand, it is some kind of a extension of nutch which seems to have
nothing to do with my question. Even if it does, it seems to be a overkill...
________________________________
From: "Arkadi.Kosmynin@csiro.au" <Ar...@csiro.au>
To: user@nutch.apache.org
Sent: Mon, July 19, 2010 9:46:03 PM
Subject: RE: mysql
Hi Savannah,
>-----Original Message-----
>From: Savannah Beckett [mailto:savannah_beckett30@yahoo.com]
>Sent: Tuesday, July 20, 2010 2:42 PM
>To: user@nutch.apache.org
>Subject: mysql
>
>Hi,
> I use DriverManager.getConnection to connect to my mysql db and do
>query in my
>plugin that extend indexfilter. It seems that indexfilter is run for
>every url
>being indexed. It means that I have to open and close connection to
>mysql db
>each time a url is being indexed, it is unefficient. Is there a way to
>open and
>close connection to mysql db only once for all urls, not for each url?
Yes, there is. See Arch index filter:
http://www.atnf.csiro.au/computing/software/arch/
Regards,
Arkadi
>
>Thanks.
>
>
>
RE: mysql
Posted by Ar...@csiro.au.
Hi Savannah,
>-----Original Message-----
>From: Savannah Beckett [mailto:savannah_beckett30@yahoo.com]
>Sent: Tuesday, July 20, 2010 2:42 PM
>To: user@nutch.apache.org
>Subject: mysql
>
>Hi,
> I use DriverManager.getConnection to connect to my mysql db and do
>query in my
>plugin that extend indexfilter. It seems that indexfilter is run for
>every url
>being indexed. It means that I have to open and close connection to
>mysql db
>each time a url is being indexed, it is unefficient. Is there a way to
>open and
>close connection to mysql db only once for all urls, not for each url?
Yes, there is. See Arch index filter:
http://www.atnf.csiro.au/computing/software/arch/
Regards,
Arkadi
>
>Thanks.
>
>
>