You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2012/02/11 23:21:54 UTC
Understanding NutchConfigration properly
Hi,
When specifying configurations for Hadoop, we are actually for using
NutchConfiguration to explicitly set configuration values initially loaded
from nutch-default which are overridden by nutch-site.xml.
Can someone explain where we are using the XSLTs in conf/configuration &
nutch-conf.xsl respectively and where the xslt processing is done.. There
is a missing link here for me which I would like to understand.
Thanks
--
*Lewis*
Re: Understanding NutchConfigration properly
Posted by Julien Nioche <li...@gmail.com>.
Hi Lewis,
Apart from making the config files look pretty when displayed by a browser
I can't really think of a use for these XSLT files. There might have been
another reason for them at some point but I am not aware of it
Julien
On 11 February 2012 22:21, Lewis John Mcgibbney
<le...@gmail.com>wrote:
> Hi,
>
> When specifying configurations for Hadoop, we are actually for using
> NutchConfiguration to explicitly set configuration values initially loaded
> from nutch-default which are overridden by nutch-site.xml.
>
> Can someone explain where we are using the XSLTs in conf/configuration &
> nutch-conf.xsl respectively and where the xslt processing is done.. There
> is a missing link here for me which I would like to understand.
>
> Thanks
>
> --
> *Lewis*
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Re: Understanding NutchConfigration properly
Posted by remi tassing <ta...@gmail.com>.
if they are really useless why keep them?
Remi
On Sunday, February 12, 2012, Julien Nioche <li...@gmail.com>
wrote:
> i meant bothering to remove these files not open a jira
>
> Julien
>
> On Sunday, 12 February 2012, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com>
> wrote:
>> I'm in an airport in Prague... some long boring hours until flight to
>> Edinburgh and needed some time to kill... but you're right it's not worth
>> it.
>>
>> I'll patch trunk and nutchgora, test and commit.
>>
>> Thanks
>>
>
>
>> On Sun, Feb 12, 2012 at 5:05 PM, Julien Nioche <
>> lists.digitalpebble@gmail.com> wrote:
>>
>>> Is it really worth bothering?
>>>
>>> On 12 February 2012 17:04, Lewis John Mcgibbney
>>> <le...@gmail.com>wrote:
>>>
>>> > I see a Jira ticket coming up here ...
>>> >
>>> > I'll open one up.
>>> >
>>> > Thanks
>>> >
>>> > Lewis
>>> >
>>> > On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org>
>>> wrote:
>>> >
>>> > > The xsl, xsd and dtd files are not used by Nutch anymore.
>>> > >
>>> > > > Hi,
>>> > > >
>>> > > > When specifying configurations for Hadoop, we are actually for
> using
>>> > > > NutchConfiguration to explicitly set configuration values
initially
>>> > > loaded
>>> > > > from nutch-default which are overridden by nutch-site.xml.
>>> > > >
>>> > > > Can someone explain where we are using the XSLTs in
>>> conf/configuration
>>> > &
>>> > > > nutch-conf.xsl respectively and where the xslt processing is
done..
>>> > There
>>> > > > is a missing link here for me which I would like to understand.
>>> > > >
>>> > > > Thanks
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > *Lewis*
>>> >
>>>
>>>
>>>
>>> --
>>> *
>>> *Open Source Solutions for Text Engineering
>>>
>>> http://digitalpebble.blogspot.com/
>>> http://www.digitalpebble.com
>>> http://twitter.com/digitalpebble
>>>
>>
>>
>>
>> --
>> *Lewis*
>>
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>
Re: Understanding NutchConfigration properly
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Yeah OK. I'll have a think and deal with this @ some stage in near future.
On Mon, Feb 13, 2012 at 2:26 PM, Julien Nioche <
lists.digitalpebble@gmail.com> wrote:
>
>
> domain-suffixes.xsd seems to be used indeed.
>
Thanks for confirming
> Sure. The 2 xsl files configuration.xsl and nutch-conf.xsl seems to be
> extremely similar to each other. Again they were probably not very useful
> anymore. if you decide to get rid of them they are referenced from the conf
> files so you'd have to modify them as well
>
> Lewis
Re: Understanding NutchConfigration properly
Posted by Julien Nioche <li...@gmail.com>.
Hi Lewis,
On second inspection, it would appear that the XSD in conf/ is utilised by
> o.a.n.util.domain.DomainSuffixesReader.java?
>
domain-suffixes.xsd seems to be used indeed.
>
> I am not particularly bothered about removing them either to be honest, I
> just wanted to know exactly what was going on.
>
Sure. The 2 xsl files configuration.xsl and nutch-conf.xsl seems to be
extremely similar to each other. Again they were probably not very useful
anymore. if you decide to get rid of them they are referenced from the conf
files so you'd have to modify them as well
J.
>
> Thanks
>
>
> On Sun, Feb 12, 2012 at 5:48 PM, Julien Nioche <
> lists.digitalpebble@gmail.com> wrote:
>
>> i meant bothering to remove these files not open a jira
>>
>> Julien
>>
>> On Sunday, 12 February 2012, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com>
>> wrote:
>> > I'm in an airport in Prague... some long boring hours until flight to
>> > Edinburgh and needed some time to kill... but you're right it's not
>> worth
>> > it.
>> >
>> > I'll patch trunk and nutchgora, test and commit.
>> >
>> > Thanks
>> >
>>
>>
>> > On Sun, Feb 12, 2012 at 5:05 PM, Julien Nioche <
>> > lists.digitalpebble@gmail.com> wrote:
>> >
>> >> Is it really worth bothering?
>> >>
>> >> On 12 February 2012 17:04, Lewis John Mcgibbney
>> >> <le...@gmail.com>wrote:
>> >>
>> >> > I see a Jira ticket coming up here ...
>> >> >
>> >> > I'll open one up.
>> >> >
>> >> > Thanks
>> >> >
>> >> > Lewis
>> >> >
>> >> > On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org>
>> >> wrote:
>> >> >
>> >> > > The xsl, xsd and dtd files are not used by Nutch anymore.
>> >> > >
>> >> > > > Hi,
>> >> > > >
>> >> > > > When specifying configurations for Hadoop, we are actually for
>> using
>> >> > > > NutchConfiguration to explicitly set configuration values
>> initially
>> >> > > loaded
>> >> > > > from nutch-default which are overridden by nutch-site.xml.
>> >> > > >
>> >> > > > Can someone explain where we are using the XSLTs in
>> >> conf/configuration
>> >> > &
>> >> > > > nutch-conf.xsl respectively and where the xslt processing is
>> done..
>> >> > There
>> >> > > > is a missing link here for me which I would like to understand.
>> >> > > >
>> >> > > > Thanks
>> >> > >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > *Lewis*
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> *
>> >> *Open Source Solutions for Text Engineering
>> >>
>> >> http://digitalpebble.blogspot.com/
>> >> http://www.digitalpebble.com
>> >> http://twitter.com/digitalpebble
>> >>
>> >
>> >
>> >
>> > --
>> > *Lewis*
>> >
>>
>> --
>> *
>> *Open Source Solutions for Text Engineering
>>
>> http://digitalpebble.blogspot.com/
>> http://www.digitalpebble.com
>> http://twitter.com/digitalpebble
>>
>
>
>
> --
> *Lewis*
>
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Re: Understanding NutchConfigration properly
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Julien,
On second inspection, it would appear that the XSD in conf/ is utilised by
o.a.n.util.domain.DomainSuffixesReader.java?
I am not particularly bothered about removing them either to be honest, I
just wanted to know exactly what was going on.
Thanks
On Sun, Feb 12, 2012 at 5:48 PM, Julien Nioche <
lists.digitalpebble@gmail.com> wrote:
> i meant bothering to remove these files not open a jira
>
> Julien
>
> On Sunday, 12 February 2012, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com>
> wrote:
> > I'm in an airport in Prague... some long boring hours until flight to
> > Edinburgh and needed some time to kill... but you're right it's not worth
> > it.
> >
> > I'll patch trunk and nutchgora, test and commit.
> >
> > Thanks
> >
>
>
> > On Sun, Feb 12, 2012 at 5:05 PM, Julien Nioche <
> > lists.digitalpebble@gmail.com> wrote:
> >
> >> Is it really worth bothering?
> >>
> >> On 12 February 2012 17:04, Lewis John Mcgibbney
> >> <le...@gmail.com>wrote:
> >>
> >> > I see a Jira ticket coming up here ...
> >> >
> >> > I'll open one up.
> >> >
> >> > Thanks
> >> >
> >> > Lewis
> >> >
> >> > On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org>
> >> wrote:
> >> >
> >> > > The xsl, xsd and dtd files are not used by Nutch anymore.
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > When specifying configurations for Hadoop, we are actually for
> using
> >> > > > NutchConfiguration to explicitly set configuration values
> initially
> >> > > loaded
> >> > > > from nutch-default which are overridden by nutch-site.xml.
> >> > > >
> >> > > > Can someone explain where we are using the XSLTs in
> >> conf/configuration
> >> > &
> >> > > > nutch-conf.xsl respectively and where the xslt processing is
> done..
> >> > There
> >> > > > is a missing link here for me which I would like to understand.
> >> > > >
> >> > > > Thanks
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > *Lewis*
> >> >
> >>
> >>
> >>
> >> --
> >> *
> >> *Open Source Solutions for Text Engineering
> >>
> >> http://digitalpebble.blogspot.com/
> >> http://www.digitalpebble.com
> >> http://twitter.com/digitalpebble
> >>
> >
> >
> >
> > --
> > *Lewis*
> >
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>
--
*Lewis*
Re: Understanding NutchConfigration properly
Posted by Julien Nioche <li...@gmail.com>.
i meant bothering to remove these files not open a jira
Julien
On Sunday, 12 February 2012, Lewis John Mcgibbney <le...@gmail.com>
wrote:
> I'm in an airport in Prague... some long boring hours until flight to
> Edinburgh and needed some time to kill... but you're right it's not worth
> it.
>
> I'll patch trunk and nutchgora, test and commit.
>
> Thanks
>
> On Sun, Feb 12, 2012 at 5:05 PM, Julien Nioche <
> lists.digitalpebble@gmail.com> wrote:
>
>> Is it really worth bothering?
>>
>> On 12 February 2012 17:04, Lewis John Mcgibbney
>> <le...@gmail.com>wrote:
>>
>> > I see a Jira ticket coming up here ...
>> >
>> > I'll open one up.
>> >
>> > Thanks
>> >
>> > Lewis
>> >
>> > On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org>
>> wrote:
>> >
>> > > The xsl, xsd and dtd files are not used by Nutch anymore.
>> > >
>> > > > Hi,
>> > > >
>> > > > When specifying configurations for Hadoop, we are actually for
using
>> > > > NutchConfiguration to explicitly set configuration values initially
>> > > loaded
>> > > > from nutch-default which are overridden by nutch-site.xml.
>> > > >
>> > > > Can someone explain where we are using the XSLTs in
>> conf/configuration
>> > &
>> > > > nutch-conf.xsl respectively and where the xslt processing is done..
>> > There
>> > > > is a missing link here for me which I would like to understand.
>> > > >
>> > > > Thanks
>> > >
>> >
>> >
>> >
>> > --
>> > *Lewis*
>> >
>>
>>
>>
>> --
>> *
>> *Open Source Solutions for Text Engineering
>>
>> http://digitalpebble.blogspot.com/
>> http://www.digitalpebble.com
>> http://twitter.com/digitalpebble
>>
>
>
>
> --
> *Lewis*
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Re: Understanding NutchConfigration properly
Posted by Lewis John Mcgibbney <le...@gmail.com>.
I'm in an airport in Prague... some long boring hours until flight to
Edinburgh and needed some time to kill... but you're right it's not worth
it.
I'll patch trunk and nutchgora, test and commit.
Thanks
On Sun, Feb 12, 2012 at 5:05 PM, Julien Nioche <
lists.digitalpebble@gmail.com> wrote:
> Is it really worth bothering?
>
> On 12 February 2012 17:04, Lewis John Mcgibbney
> <le...@gmail.com>wrote:
>
> > I see a Jira ticket coming up here ...
> >
> > I'll open one up.
> >
> > Thanks
> >
> > Lewis
> >
> > On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org>
> wrote:
> >
> > > The xsl, xsd and dtd files are not used by Nutch anymore.
> > >
> > > > Hi,
> > > >
> > > > When specifying configurations for Hadoop, we are actually for using
> > > > NutchConfiguration to explicitly set configuration values initially
> > > loaded
> > > > from nutch-default which are overridden by nutch-site.xml.
> > > >
> > > > Can someone explain where we are using the XSLTs in
> conf/configuration
> > &
> > > > nutch-conf.xsl respectively and where the xslt processing is done..
> > There
> > > > is a missing link here for me which I would like to understand.
> > > >
> > > > Thanks
> > >
> >
> >
> >
> > --
> > *Lewis*
> >
>
>
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>
--
*Lewis*
Re: Understanding NutchConfigration properly
Posted by Julien Nioche <li...@gmail.com>.
Is it really worth bothering?
On 12 February 2012 17:04, Lewis John Mcgibbney
<le...@gmail.com>wrote:
> I see a Jira ticket coming up here ...
>
> I'll open one up.
>
> Thanks
>
> Lewis
>
> On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org> wrote:
>
> > The xsl, xsd and dtd files are not used by Nutch anymore.
> >
> > > Hi,
> > >
> > > When specifying configurations for Hadoop, we are actually for using
> > > NutchConfiguration to explicitly set configuration values initially
> > loaded
> > > from nutch-default which are overridden by nutch-site.xml.
> > >
> > > Can someone explain where we are using the XSLTs in conf/configuration
> &
> > > nutch-conf.xsl respectively and where the xslt processing is done..
> There
> > > is a missing link here for me which I would like to understand.
> > >
> > > Thanks
> >
>
>
>
> --
> *Lewis*
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Re: Understanding NutchConfigration properly
Posted by Lewis John Mcgibbney <le...@gmail.com>.
I see a Jira ticket coming up here ...
I'll open one up.
Thanks
Lewis
On Sat, Feb 11, 2012 at 10:58 PM, Markus Jelsma <ma...@apache.org> wrote:
> The xsl, xsd and dtd files are not used by Nutch anymore.
>
> > Hi,
> >
> > When specifying configurations for Hadoop, we are actually for using
> > NutchConfiguration to explicitly set configuration values initially
> loaded
> > from nutch-default which are overridden by nutch-site.xml.
> >
> > Can someone explain where we are using the XSLTs in conf/configuration &
> > nutch-conf.xsl respectively and where the xslt processing is done.. There
> > is a missing link here for me which I would like to understand.
> >
> > Thanks
>
--
*Lewis*
Re: Understanding NutchConfigration properly
Posted by Markus Jelsma <ma...@apache.org>.
The xsl, xsd and dtd files are not used by Nutch anymore.
> Hi,
>
> When specifying configurations for Hadoop, we are actually for using
> NutchConfiguration to explicitly set configuration values initially loaded
> from nutch-default which are overridden by nutch-site.xml.
>
> Can someone explain where we are using the XSLTs in conf/configuration &
> nutch-conf.xsl respectively and where the xslt processing is done.. There
> is a missing link here for me which I would like to understand.
>
> Thanks