You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Andrew Pennebaker <ap...@42six.com> on 2013/08/16 15:35:16 UTC
Automating nutch installation
I think it would make nutch installation easier if we released standardized
packages.
For example, the Nutch tutorial calls for manually downloading a ZIP,
extracting the contents, installing ant, compiling nutch with ant, and
adjusting PATH.
What if instead, Ubuntu users could simply "apt-get install nutch", the
same way you can install wget with "apt-get install wget"?
Similarly, we could help out Mac developers by submitting a
Homebrew<http://brew.sh/> formula
for nutch, and Windows developers would enjoy a
Chocolatey<http://chocolatey.org/> package.
The easier the installation process, the more people want to use nutch!
Re: Automating nutch installation
Posted by Andrew Pennebaker <ap...@42six.com>.
Done.
https://issues.apache.org/jira/browse/NUTCH-1626
https://issues.apache.org/jira/browse/NUTCH-1627
https://issues.apache.org/jira/browse/NUTCH-1628
On Mon, Aug 19, 2013 at 3:03 PM, Nicholas Roberts <niccolo.roberts@gmail.com
> wrote:
> I'm an Ubuntu user nowadays, no brew help ... yes, a Jira issue is a good
> idea
>
>
> On Mon, Aug 19, 2013 at 12:00 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > I suggest that this moves on to the relevant Jira issue. If we want to
> make
> > Debian or other packages for Nutch then this is certainly a task which
> we
> > should move to dev@ and Jira.
> > This is great
> >
> > On Monday, August 19, 2013, Andrew Pennebaker <ap...@42six.com>
> > wrote:
> > > I'm piecing together a Homebrew formula for install nutch on Macs. The
> > `ant
> > > runtime` build step succeeds, but for some reason, the resulting
> runtime/
> > > directory isn't kept in the Cellar. Would someone kindly help debug
> this?
> > >
> > > https://gist.github.com/apennebaker/6270993
> > >
> > >
> > > On Mon, Aug 19, 2013 at 12:37 PM, Andrew Pennebaker
> > > <ap...@42six.com>wrote:
> > >
> > >> Hmm. Does nutch offer a command line option to specify a configuration
> > >> file? That way, you could `nutch -c coolsite.xml && nutch -c
> > >> awesomesite.xml`, for example, and you could refer to a single nutch
> > binary
> > >> installed globally.
> > >>
> > >>
> > >> On Mon, Aug 19, 2013 at 12:27 PM, Nicholas Roberts <
> > >> niccolo.roberts@gmail.com> wrote:
> > >>
> > >>> I understand
> > >>>
> > >>> but how will an installer handle the use case which is
> > >>>
> > >>> "install nutch with multiple configurations for numerous crawls?"
> > >>>
> > >>> basically, for me anyway, that means multiple installations of nutch
> > via
> > >>> ant/ant job
> > >>>
> > >>> I'm kludging a kind of apache solr multicore for nutch (not very
> welll
> > I
> > >>> suspect)
> > >>>
> > >>>
> > >>> On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <
> > apennebaker@42six.com
> > >>> >wrote:
> > >>>
> > >>> > A bash script would be a great start! Though systems that depend on
> > >>> nutch
> > >>> > would want to handle it via a standard package manager, so that's
> the
> > >>> end
> > >>> > goal.
> > >>> >
> > >>> >
> > >>> > On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
> > >>> > niccolo.roberts@gmail.com> wrote:
> > >>> >
> > >>> > > wouldn't a bash script cover mac and linux?
> > >>> > >
> > >>> > > also, nutch doesn't do multi-core like solr so how do you install
> > many
> > >>> > > variations?
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
> > >>> > > lewis.mcgibbney@gmail.com> wrote:
> > >>> > >
> > >>> > > > Hi Andrew,
> > >>> > > > It seems that from the core Nutch team the time and desire is
> not
> > >>> there
> > >>> > > > right now to push forward with your proposals.
> > >>> > > > This DOES NOT mean that the proposals will be ignored.
> > >>> > > > I would FULLY back convenient packaging for Nutch artifacts...
> it
> > >>> would
> > >>> > > be
> > >>> > > > a really great community contributions and we would be one of
> the
> > >>> few
> > >>> > > > Apache projects that actually could claim to have such facets
> to
> > the
> > >>> > > > community.
> > >>> > > > Please, feel free to press on with this if you have the time...
> > >>> right
> > >>> > > now I
> > >>> > > > certainly do not.
> > >>> > > > Thanks
> > >>> > > > Lewis
> > >>> > > >
> > >>> > > >
> > >>> > > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
> > >>> > > apennebaker@42six.com
> > >>> > > > >wrote:
> > >>> > > >
> > >>> > > > > I've built a .deb for Digital Paintball a few years ago (
> > >>> > > > >
> > >>> >
> > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> > >>> > > > >
> > >>> http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> > >>> > > > > There's a tutorial for doing this:
> > >>> > > > >
> > >>> > > > > https://wiki.debian.org/IntroDebianPackaging
> > >>> > > > >
> > >>> > > > > However, I'm using nutch for work, and may have an awkward
> time
> > >>> > > charging
> > >>> > > > > hours for contributing to nutch. If we could work together to
> > get
> > >>> a
> > >>> > > .deb,
> > >>> > > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula
> for
> > >>> Mac,
> > >>> > > and
> > >>> > > > a
> > >>> > > > > Chocolatey package for Windows, that would make my life, and
> > >>> probably
> > >
> >
> > --
> > *Lewis*
> >
>
>
>
> --
>
> --
> Nicholas Roberts
> US 510-684-8264
> http://Permaculture.TV <http://permaculture.tv/>
>
Re: Automating nutch installation
Posted by Nicholas Roberts <ni...@gmail.com>.
I'm an Ubuntu user nowadays, no brew help ... yes, a Jira issue is a good
idea
On Mon, Aug 19, 2013 at 12:00 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> I suggest that this moves on to the relevant Jira issue. If we want to make
> Debian or other packages for Nutch then this is certainly a task which we
> should move to dev@ and Jira.
> This is great
>
> On Monday, August 19, 2013, Andrew Pennebaker <ap...@42six.com>
> wrote:
> > I'm piecing together a Homebrew formula for install nutch on Macs. The
> `ant
> > runtime` build step succeeds, but for some reason, the resulting runtime/
> > directory isn't kept in the Cellar. Would someone kindly help debug this?
> >
> > https://gist.github.com/apennebaker/6270993
> >
> >
> > On Mon, Aug 19, 2013 at 12:37 PM, Andrew Pennebaker
> > <ap...@42six.com>wrote:
> >
> >> Hmm. Does nutch offer a command line option to specify a configuration
> >> file? That way, you could `nutch -c coolsite.xml && nutch -c
> >> awesomesite.xml`, for example, and you could refer to a single nutch
> binary
> >> installed globally.
> >>
> >>
> >> On Mon, Aug 19, 2013 at 12:27 PM, Nicholas Roberts <
> >> niccolo.roberts@gmail.com> wrote:
> >>
> >>> I understand
> >>>
> >>> but how will an installer handle the use case which is
> >>>
> >>> "install nutch with multiple configurations for numerous crawls?"
> >>>
> >>> basically, for me anyway, that means multiple installations of nutch
> via
> >>> ant/ant job
> >>>
> >>> I'm kludging a kind of apache solr multicore for nutch (not very welll
> I
> >>> suspect)
> >>>
> >>>
> >>> On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <
> apennebaker@42six.com
> >>> >wrote:
> >>>
> >>> > A bash script would be a great start! Though systems that depend on
> >>> nutch
> >>> > would want to handle it via a standard package manager, so that's the
> >>> end
> >>> > goal.
> >>> >
> >>> >
> >>> > On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
> >>> > niccolo.roberts@gmail.com> wrote:
> >>> >
> >>> > > wouldn't a bash script cover mac and linux?
> >>> > >
> >>> > > also, nutch doesn't do multi-core like solr so how do you install
> many
> >>> > > variations?
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
> >>> > > lewis.mcgibbney@gmail.com> wrote:
> >>> > >
> >>> > > > Hi Andrew,
> >>> > > > It seems that from the core Nutch team the time and desire is not
> >>> there
> >>> > > > right now to push forward with your proposals.
> >>> > > > This DOES NOT mean that the proposals will be ignored.
> >>> > > > I would FULLY back convenient packaging for Nutch artifacts... it
> >>> would
> >>> > > be
> >>> > > > a really great community contributions and we would be one of the
> >>> few
> >>> > > > Apache projects that actually could claim to have such facets to
> the
> >>> > > > community.
> >>> > > > Please, feel free to press on with this if you have the time...
> >>> right
> >>> > > now I
> >>> > > > certainly do not.
> >>> > > > Thanks
> >>> > > > Lewis
> >>> > > >
> >>> > > >
> >>> > > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
> >>> > > apennebaker@42six.com
> >>> > > > >wrote:
> >>> > > >
> >>> > > > > I've built a .deb for Digital Paintball a few years ago (
> >>> > > > >
> >>> >
> http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> >>> > > > >
> >>> http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> >>> > > > > There's a tutorial for doing this:
> >>> > > > >
> >>> > > > > https://wiki.debian.org/IntroDebianPackaging
> >>> > > > >
> >>> > > > > However, I'm using nutch for work, and may have an awkward time
> >>> > > charging
> >>> > > > > hours for contributing to nutch. If we could work together to
> get
> >>> a
> >>> > > .deb,
> >>> > > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for
> >>> Mac,
> >>> > > and
> >>> > > > a
> >>> > > > > Chocolatey package for Windows, that would make my life, and
> >>> probably
> >
>
> --
> *Lewis*
>
--
--
Nicholas Roberts
US 510-684-8264
http://Permaculture.TV <http://permaculture.tv/>
Re: Automating nutch installation
Posted by Lewis John Mcgibbney <le...@gmail.com>.
I suggest that this moves on to the relevant Jira issue. If we want to make
Debian or other packages for Nutch then this is certainly a task which we
should move to dev@ and Jira.
This is great
On Monday, August 19, 2013, Andrew Pennebaker <ap...@42six.com> wrote:
> I'm piecing together a Homebrew formula for install nutch on Macs. The
`ant
> runtime` build step succeeds, but for some reason, the resulting runtime/
> directory isn't kept in the Cellar. Would someone kindly help debug this?
>
> https://gist.github.com/apennebaker/6270993
>
>
> On Mon, Aug 19, 2013 at 12:37 PM, Andrew Pennebaker
> <ap...@42six.com>wrote:
>
>> Hmm. Does nutch offer a command line option to specify a configuration
>> file? That way, you could `nutch -c coolsite.xml && nutch -c
>> awesomesite.xml`, for example, and you could refer to a single nutch
binary
>> installed globally.
>>
>>
>> On Mon, Aug 19, 2013 at 12:27 PM, Nicholas Roberts <
>> niccolo.roberts@gmail.com> wrote:
>>
>>> I understand
>>>
>>> but how will an installer handle the use case which is
>>>
>>> "install nutch with multiple configurations for numerous crawls?"
>>>
>>> basically, for me anyway, that means multiple installations of nutch via
>>> ant/ant job
>>>
>>> I'm kludging a kind of apache solr multicore for nutch (not very welll I
>>> suspect)
>>>
>>>
>>> On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <
apennebaker@42six.com
>>> >wrote:
>>>
>>> > A bash script would be a great start! Though systems that depend on
>>> nutch
>>> > would want to handle it via a standard package manager, so that's the
>>> end
>>> > goal.
>>> >
>>> >
>>> > On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
>>> > niccolo.roberts@gmail.com> wrote:
>>> >
>>> > > wouldn't a bash script cover mac and linux?
>>> > >
>>> > > also, nutch doesn't do multi-core like solr so how do you install
many
>>> > > variations?
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
>>> > > lewis.mcgibbney@gmail.com> wrote:
>>> > >
>>> > > > Hi Andrew,
>>> > > > It seems that from the core Nutch team the time and desire is not
>>> there
>>> > > > right now to push forward with your proposals.
>>> > > > This DOES NOT mean that the proposals will be ignored.
>>> > > > I would FULLY back convenient packaging for Nutch artifacts... it
>>> would
>>> > > be
>>> > > > a really great community contributions and we would be one of the
>>> few
>>> > > > Apache projects that actually could claim to have such facets to
the
>>> > > > community.
>>> > > > Please, feel free to press on with this if you have the time...
>>> right
>>> > > now I
>>> > > > certainly do not.
>>> > > > Thanks
>>> > > > Lewis
>>> > > >
>>> > > >
>>> > > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
>>> > > apennebaker@42six.com
>>> > > > >wrote:
>>> > > >
>>> > > > > I've built a .deb for Digital Paintball a few years ago (
>>> > > > >
>>> > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
>>> > > > >
>>> http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
>>> > > > > There's a tutorial for doing this:
>>> > > > >
>>> > > > > https://wiki.debian.org/IntroDebianPackaging
>>> > > > >
>>> > > > > However, I'm using nutch for work, and may have an awkward time
>>> > > charging
>>> > > > > hours for contributing to nutch. If we could work together to
get
>>> a
>>> > > .deb,
>>> > > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for
>>> Mac,
>>> > > and
>>> > > > a
>>> > > > > Chocolatey package for Windows, that would make my life, and
>>> probably
>
--
*Lewis*
Re: Automating nutch installation
Posted by Andrew Pennebaker <ap...@42six.com>.
I'm piecing together a Homebrew formula for install nutch on Macs. The `ant
runtime` build step succeeds, but for some reason, the resulting runtime/
directory isn't kept in the Cellar. Would someone kindly help debug this?
https://gist.github.com/apennebaker/6270993
On Mon, Aug 19, 2013 at 12:37 PM, Andrew Pennebaker
<ap...@42six.com>wrote:
> Hmm. Does nutch offer a command line option to specify a configuration
> file? That way, you could `nutch -c coolsite.xml && nutch -c
> awesomesite.xml`, for example, and you could refer to a single nutch binary
> installed globally.
>
>
> On Mon, Aug 19, 2013 at 12:27 PM, Nicholas Roberts <
> niccolo.roberts@gmail.com> wrote:
>
>> I understand
>>
>> but how will an installer handle the use case which is
>>
>> "install nutch with multiple configurations for numerous crawls?"
>>
>> basically, for me anyway, that means multiple installations of nutch via
>> ant/ant job
>>
>> I'm kludging a kind of apache solr multicore for nutch (not very welll I
>> suspect)
>>
>>
>> On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <apennebaker@42six.com
>> >wrote:
>>
>> > A bash script would be a great start! Though systems that depend on
>> nutch
>> > would want to handle it via a standard package manager, so that's the
>> end
>> > goal.
>> >
>> >
>> > On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
>> > niccolo.roberts@gmail.com> wrote:
>> >
>> > > wouldn't a bash script cover mac and linux?
>> > >
>> > > also, nutch doesn't do multi-core like solr so how do you install many
>> > > variations?
>> > >
>> > >
>> > >
>> > >
>> > > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
>> > > lewis.mcgibbney@gmail.com> wrote:
>> > >
>> > > > Hi Andrew,
>> > > > It seems that from the core Nutch team the time and desire is not
>> there
>> > > > right now to push forward with your proposals.
>> > > > This DOES NOT mean that the proposals will be ignored.
>> > > > I would FULLY back convenient packaging for Nutch artifacts... it
>> would
>> > > be
>> > > > a really great community contributions and we would be one of the
>> few
>> > > > Apache projects that actually could claim to have such facets to the
>> > > > community.
>> > > > Please, feel free to press on with this if you have the time...
>> right
>> > > now I
>> > > > certainly do not.
>> > > > Thanks
>> > > > Lewis
>> > > >
>> > > >
>> > > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
>> > > apennebaker@42six.com
>> > > > >wrote:
>> > > >
>> > > > > I've built a .deb for Digital Paintball a few years ago (
>> > > > >
>> > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
>> > > > >
>> http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
>> > > > > There's a tutorial for doing this:
>> > > > >
>> > > > > https://wiki.debian.org/IntroDebianPackaging
>> > > > >
>> > > > > However, I'm using nutch for work, and may have an awkward time
>> > > charging
>> > > > > hours for contributing to nutch. If we could work together to get
>> a
>> > > .deb,
>> > > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for
>> Mac,
>> > > and
>> > > > a
>> > > > > Chocolatey package for Windows, that would make my life, and
>> probably
>> > > > > thousands of others' lives, a whole lot easier. I'll see what I
>> can
>> > get
>> > > > > done during downtime, probably prioritizing Homebrew first.
>> > > > >
>> > > > >
>> > > > > On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
>> > > > > lewis.mcgibbney@gmail.com> wrote:
>> > > > >
>> > > > > > Hi Andrew,
>> > > > > > Have you written and maintained Debian packages before?
>> > > > > > I opened a Jira issue for this a whiel back and as far as I
>> know it
>> > > is
>> > > > a
>> > > > > > far from trivial process but I would be very very interested to
>> > get a
>> > > > > > Debiran package for Nutch and look at a mac package if so
>> required.
>> > > > > > There is an issue of who is going to maintain these packages?
>> > > > > > Thanks
>> > > > > > lewis
>> > > > > >
>> > > > > > On Friday, August 16, 2013, Andrew Pennebaker <
>> > apennebaker@42six.com
>> > > >
>> > > > > > wrote:
>> > > > > > > I think it would make nutch installation easier if we released
>> > > > > > standardized
>> > > > > > > packages.
>> > > > > > >
>> > > > > > > For example, the Nutch tutorial calls for manually
>> downloading a
>> > > ZIP,
>> > > > > > > extracting the contents, installing ant, compiling nutch with
>> > ant,
>> > > > and
>> > > > > > > adjusting PATH.
>> > > > > > >
>> > > > > > > What if instead, Ubuntu users could simply "apt-get install
>> > nutch",
>> > > > the
>> > > > > > > same way you can install wget with "apt-get install wget"?
>> > > > > > >
>> > > > > > > Similarly, we could help out Mac developers by submitting a
>> > > > > > > Homebrew<http://brew.sh/> formula
>> > > > > > > for nutch, and Windows developers would enjoy a
>> > > > > > > Chocolatey<http://chocolatey.org/> package.
>> > > > > > > The easier the installation process, the more people want to
>> use
>> > > > nutch!
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > *Lewis*
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Lewis*
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > >
>> > > --
>> > > Nicholas Roberts
>> > > US 510-684-8264
>> > > http://Permaculture.TV <http://permaculture.tv/>
>> > >
>> >
>>
>>
>>
>> --
>>
>> --
>> Nicholas Roberts
>> US 510-684-8264
>> http://Permaculture.TV <http://permaculture.tv/>
>>
>
>
Re: Automating nutch installation
Posted by Andrew Pennebaker <ap...@42six.com>.
Hmm. Does nutch offer a command line option to specify a configuration
file? That way, you could `nutch -c coolsite.xml && nutch -c
awesomesite.xml`, for example, and you could refer to a single nutch binary
installed globally.
On Mon, Aug 19, 2013 at 12:27 PM, Nicholas Roberts <
niccolo.roberts@gmail.com> wrote:
> I understand
>
> but how will an installer handle the use case which is
>
> "install nutch with multiple configurations for numerous crawls?"
>
> basically, for me anyway, that means multiple installations of nutch via
> ant/ant job
>
> I'm kludging a kind of apache solr multicore for nutch (not very welll I
> suspect)
>
>
> On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <apennebaker@42six.com
> >wrote:
>
> > A bash script would be a great start! Though systems that depend on nutch
> > would want to handle it via a standard package manager, so that's the end
> > goal.
> >
> >
> > On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
> > niccolo.roberts@gmail.com> wrote:
> >
> > > wouldn't a bash script cover mac and linux?
> > >
> > > also, nutch doesn't do multi-core like solr so how do you install many
> > > variations?
> > >
> > >
> > >
> > >
> > > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
> > > lewis.mcgibbney@gmail.com> wrote:
> > >
> > > > Hi Andrew,
> > > > It seems that from the core Nutch team the time and desire is not
> there
> > > > right now to push forward with your proposals.
> > > > This DOES NOT mean that the proposals will be ignored.
> > > > I would FULLY back convenient packaging for Nutch artifacts... it
> would
> > > be
> > > > a really great community contributions and we would be one of the few
> > > > Apache projects that actually could claim to have such facets to the
> > > > community.
> > > > Please, feel free to press on with this if you have the time... right
> > > now I
> > > > certainly do not.
> > > > Thanks
> > > > Lewis
> > > >
> > > >
> > > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
> > > apennebaker@42six.com
> > > > >wrote:
> > > >
> > > > > I've built a .deb for Digital Paintball a few years ago (
> > > > >
> > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> > > > > http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb
> ).
> > > > > There's a tutorial for doing this:
> > > > >
> > > > > https://wiki.debian.org/IntroDebianPackaging
> > > > >
> > > > > However, I'm using nutch for work, and may have an awkward time
> > > charging
> > > > > hours for contributing to nutch. If we could work together to get a
> > > .deb,
> > > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for
> Mac,
> > > and
> > > > a
> > > > > Chocolatey package for Windows, that would make my life, and
> probably
> > > > > thousands of others' lives, a whole lot easier. I'll see what I can
> > get
> > > > > done during downtime, probably prioritizing Homebrew first.
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
> > > > > lewis.mcgibbney@gmail.com> wrote:
> > > > >
> > > > > > Hi Andrew,
> > > > > > Have you written and maintained Debian packages before?
> > > > > > I opened a Jira issue for this a whiel back and as far as I know
> it
> > > is
> > > > a
> > > > > > far from trivial process but I would be very very interested to
> > get a
> > > > > > Debiran package for Nutch and look at a mac package if so
> required.
> > > > > > There is an issue of who is going to maintain these packages?
> > > > > > Thanks
> > > > > > lewis
> > > > > >
> > > > > > On Friday, August 16, 2013, Andrew Pennebaker <
> > apennebaker@42six.com
> > > >
> > > > > > wrote:
> > > > > > > I think it would make nutch installation easier if we released
> > > > > > standardized
> > > > > > > packages.
> > > > > > >
> > > > > > > For example, the Nutch tutorial calls for manually downloading
> a
> > > ZIP,
> > > > > > > extracting the contents, installing ant, compiling nutch with
> > ant,
> > > > and
> > > > > > > adjusting PATH.
> > > > > > >
> > > > > > > What if instead, Ubuntu users could simply "apt-get install
> > nutch",
> > > > the
> > > > > > > same way you can install wget with "apt-get install wget"?
> > > > > > >
> > > > > > > Similarly, we could help out Mac developers by submitting a
> > > > > > > Homebrew<http://brew.sh/> formula
> > > > > > > for nutch, and Windows developers would enjoy a
> > > > > > > Chocolatey<http://chocolatey.org/> package.
> > > > > > > The easier the installation process, the more people want to
> use
> > > > nutch!
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > *Lewis*
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Lewis*
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > --
> > > Nicholas Roberts
> > > US 510-684-8264
> > > http://Permaculture.TV <http://permaculture.tv/>
> > >
> >
>
>
>
> --
>
> --
> Nicholas Roberts
> US 510-684-8264
> http://Permaculture.TV <http://permaculture.tv/>
>
Re: Automating nutch installation
Posted by Nicholas Roberts <ni...@gmail.com>.
I understand
but how will an installer handle the use case which is
"install nutch with multiple configurations for numerous crawls?"
basically, for me anyway, that means multiple installations of nutch via
ant/ant job
I'm kludging a kind of apache solr multicore for nutch (not very welll I
suspect)
On Mon, Aug 19, 2013 at 8:54 AM, Andrew Pennebaker <ap...@42six.com>wrote:
> A bash script would be a great start! Though systems that depend on nutch
> would want to handle it via a standard package manager, so that's the end
> goal.
>
>
> On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
> niccolo.roberts@gmail.com> wrote:
>
> > wouldn't a bash script cover mac and linux?
> >
> > also, nutch doesn't do multi-core like solr so how do you install many
> > variations?
> >
> >
> >
> >
> > On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> > > Hi Andrew,
> > > It seems that from the core Nutch team the time and desire is not there
> > > right now to push forward with your proposals.
> > > This DOES NOT mean that the proposals will be ignored.
> > > I would FULLY back convenient packaging for Nutch artifacts... it would
> > be
> > > a really great community contributions and we would be one of the few
> > > Apache projects that actually could claim to have such facets to the
> > > community.
> > > Please, feel free to press on with this if you have the time... right
> > now I
> > > certainly do not.
> > > Thanks
> > > Lewis
> > >
> > >
> > > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
> > apennebaker@42six.com
> > > >wrote:
> > >
> > > > I've built a .deb for Digital Paintball a few years ago (
> > > >
> http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> > > > http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> > > > There's a tutorial for doing this:
> > > >
> > > > https://wiki.debian.org/IntroDebianPackaging
> > > >
> > > > However, I'm using nutch for work, and may have an awkward time
> > charging
> > > > hours for contributing to nutch. If we could work together to get a
> > .deb,
> > > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for Mac,
> > and
> > > a
> > > > Chocolatey package for Windows, that would make my life, and probably
> > > > thousands of others' lives, a whole lot easier. I'll see what I can
> get
> > > > done during downtime, probably prioritizing Homebrew first.
> > > >
> > > >
> > > > On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
> > > > lewis.mcgibbney@gmail.com> wrote:
> > > >
> > > > > Hi Andrew,
> > > > > Have you written and maintained Debian packages before?
> > > > > I opened a Jira issue for this a whiel back and as far as I know it
> > is
> > > a
> > > > > far from trivial process but I would be very very interested to
> get a
> > > > > Debiran package for Nutch and look at a mac package if so required.
> > > > > There is an issue of who is going to maintain these packages?
> > > > > Thanks
> > > > > lewis
> > > > >
> > > > > On Friday, August 16, 2013, Andrew Pennebaker <
> apennebaker@42six.com
> > >
> > > > > wrote:
> > > > > > I think it would make nutch installation easier if we released
> > > > > standardized
> > > > > > packages.
> > > > > >
> > > > > > For example, the Nutch tutorial calls for manually downloading a
> > ZIP,
> > > > > > extracting the contents, installing ant, compiling nutch with
> ant,
> > > and
> > > > > > adjusting PATH.
> > > > > >
> > > > > > What if instead, Ubuntu users could simply "apt-get install
> nutch",
> > > the
> > > > > > same way you can install wget with "apt-get install wget"?
> > > > > >
> > > > > > Similarly, we could help out Mac developers by submitting a
> > > > > > Homebrew<http://brew.sh/> formula
> > > > > > for nutch, and Windows developers would enjoy a
> > > > > > Chocolatey<http://chocolatey.org/> package.
> > > > > > The easier the installation process, the more people want to use
> > > nutch!
> > > > > >
> > > > >
> > > > > --
> > > > > *Lewis*
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *Lewis*
> > >
> >
> >
> >
> > --
> >
> > --
> > Nicholas Roberts
> > US 510-684-8264
> > http://Permaculture.TV <http://permaculture.tv/>
> >
>
--
--
Nicholas Roberts
US 510-684-8264
http://Permaculture.TV <http://permaculture.tv/>
Re: Automating nutch installation
Posted by Andrew Pennebaker <ap...@42six.com>.
A bash script would be a great start! Though systems that depend on nutch
would want to handle it via a standard package manager, so that's the end
goal.
On Mon, Aug 19, 2013 at 12:43 AM, Nicholas Roberts <
niccolo.roberts@gmail.com> wrote:
> wouldn't a bash script cover mac and linux?
>
> also, nutch doesn't do multi-core like solr so how do you install many
> variations?
>
>
>
>
> On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi Andrew,
> > It seems that from the core Nutch team the time and desire is not there
> > right now to push forward with your proposals.
> > This DOES NOT mean that the proposals will be ignored.
> > I would FULLY back convenient packaging for Nutch artifacts... it would
> be
> > a really great community contributions and we would be one of the few
> > Apache projects that actually could claim to have such facets to the
> > community.
> > Please, feel free to press on with this if you have the time... right
> now I
> > certainly do not.
> > Thanks
> > Lewis
> >
> >
> > On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <
> apennebaker@42six.com
> > >wrote:
> >
> > > I've built a .deb for Digital Paintball a few years ago (
> > > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> > > http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> > > There's a tutorial for doing this:
> > >
> > > https://wiki.debian.org/IntroDebianPackaging
> > >
> > > However, I'm using nutch for work, and may have an awkward time
> charging
> > > hours for contributing to nutch. If we could work together to get a
> .deb,
> > > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for Mac,
> and
> > a
> > > Chocolatey package for Windows, that would make my life, and probably
> > > thousands of others' lives, a whole lot easier. I'll see what I can get
> > > done during downtime, probably prioritizing Homebrew first.
> > >
> > >
> > > On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
> > > lewis.mcgibbney@gmail.com> wrote:
> > >
> > > > Hi Andrew,
> > > > Have you written and maintained Debian packages before?
> > > > I opened a Jira issue for this a whiel back and as far as I know it
> is
> > a
> > > > far from trivial process but I would be very very interested to get a
> > > > Debiran package for Nutch and look at a mac package if so required.
> > > > There is an issue of who is going to maintain these packages?
> > > > Thanks
> > > > lewis
> > > >
> > > > On Friday, August 16, 2013, Andrew Pennebaker <apennebaker@42six.com
> >
> > > > wrote:
> > > > > I think it would make nutch installation easier if we released
> > > > standardized
> > > > > packages.
> > > > >
> > > > > For example, the Nutch tutorial calls for manually downloading a
> ZIP,
> > > > > extracting the contents, installing ant, compiling nutch with ant,
> > and
> > > > > adjusting PATH.
> > > > >
> > > > > What if instead, Ubuntu users could simply "apt-get install nutch",
> > the
> > > > > same way you can install wget with "apt-get install wget"?
> > > > >
> > > > > Similarly, we could help out Mac developers by submitting a
> > > > > Homebrew<http://brew.sh/> formula
> > > > > for nutch, and Windows developers would enjoy a
> > > > > Chocolatey<http://chocolatey.org/> package.
> > > > > The easier the installation process, the more people want to use
> > nutch!
> > > > >
> > > >
> > > > --
> > > > *Lewis*
> > > >
> > >
> >
> >
> >
> > --
> > *Lewis*
> >
>
>
>
> --
>
> --
> Nicholas Roberts
> US 510-684-8264
> http://Permaculture.TV <http://permaculture.tv/>
>
Re: Automating nutch installation
Posted by Nicholas Roberts <ni...@gmail.com>.
wouldn't a bash script cover mac and linux?
also, nutch doesn't do multi-core like solr so how do you install many
variations?
On Sun, Aug 18, 2013 at 3:50 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Andrew,
> It seems that from the core Nutch team the time and desire is not there
> right now to push forward with your proposals.
> This DOES NOT mean that the proposals will be ignored.
> I would FULLY back convenient packaging for Nutch artifacts... it would be
> a really great community contributions and we would be one of the few
> Apache projects that actually could claim to have such facets to the
> community.
> Please, feel free to press on with this if you have the time... right now I
> certainly do not.
> Thanks
> Lewis
>
>
> On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <apennebaker@42six.com
> >wrote:
>
> > I've built a .deb for Digital Paintball a few years ago (
> > http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> > http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> > There's a tutorial for doing this:
> >
> > https://wiki.debian.org/IntroDebianPackaging
> >
> > However, I'm using nutch for work, and may have an awkward time charging
> > hours for contributing to nutch. If we could work together to get a .deb,
> > maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for Mac, and
> a
> > Chocolatey package for Windows, that would make my life, and probably
> > thousands of others' lives, a whole lot easier. I'll see what I can get
> > done during downtime, probably prioritizing Homebrew first.
> >
> >
> > On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> > > Hi Andrew,
> > > Have you written and maintained Debian packages before?
> > > I opened a Jira issue for this a whiel back and as far as I know it is
> a
> > > far from trivial process but I would be very very interested to get a
> > > Debiran package for Nutch and look at a mac package if so required.
> > > There is an issue of who is going to maintain these packages?
> > > Thanks
> > > lewis
> > >
> > > On Friday, August 16, 2013, Andrew Pennebaker <ap...@42six.com>
> > > wrote:
> > > > I think it would make nutch installation easier if we released
> > > standardized
> > > > packages.
> > > >
> > > > For example, the Nutch tutorial calls for manually downloading a ZIP,
> > > > extracting the contents, installing ant, compiling nutch with ant,
> and
> > > > adjusting PATH.
> > > >
> > > > What if instead, Ubuntu users could simply "apt-get install nutch",
> the
> > > > same way you can install wget with "apt-get install wget"?
> > > >
> > > > Similarly, we could help out Mac developers by submitting a
> > > > Homebrew<http://brew.sh/> formula
> > > > for nutch, and Windows developers would enjoy a
> > > > Chocolatey<http://chocolatey.org/> package.
> > > > The easier the installation process, the more people want to use
> nutch!
> > > >
> > >
> > > --
> > > *Lewis*
> > >
> >
>
>
>
> --
> *Lewis*
>
--
--
Nicholas Roberts
US 510-684-8264
http://Permaculture.TV <http://permaculture.tv/>
Re: Automating nutch installation
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Andrew,
It seems that from the core Nutch team the time and desire is not there
right now to push forward with your proposals.
This DOES NOT mean that the proposals will be ignored.
I would FULLY back convenient packaging for Nutch artifacts... it would be
a really great community contributions and we would be one of the few
Apache projects that actually could claim to have such facets to the
community.
Please, feel free to press on with this if you have the time... right now I
certainly do not.
Thanks
Lewis
On Fri, Aug 16, 2013 at 9:19 AM, Andrew Pennebaker <ap...@42six.com>wrote:
> I've built a .deb for Digital Paintball a few years ago (
> http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
> http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
> There's a tutorial for doing this:
>
> https://wiki.debian.org/IntroDebianPackaging
>
> However, I'm using nutch for work, and may have an awkward time charging
> hours for contributing to nutch. If we could work together to get a .deb,
> maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for Mac, and a
> Chocolatey package for Windows, that would make my life, and probably
> thousands of others' lives, a whole lot easier. I'll see what I can get
> done during downtime, probably prioritizing Homebrew first.
>
>
> On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi Andrew,
> > Have you written and maintained Debian packages before?
> > I opened a Jira issue for this a whiel back and as far as I know it is a
> > far from trivial process but I would be very very interested to get a
> > Debiran package for Nutch and look at a mac package if so required.
> > There is an issue of who is going to maintain these packages?
> > Thanks
> > lewis
> >
> > On Friday, August 16, 2013, Andrew Pennebaker <ap...@42six.com>
> > wrote:
> > > I think it would make nutch installation easier if we released
> > standardized
> > > packages.
> > >
> > > For example, the Nutch tutorial calls for manually downloading a ZIP,
> > > extracting the contents, installing ant, compiling nutch with ant, and
> > > adjusting PATH.
> > >
> > > What if instead, Ubuntu users could simply "apt-get install nutch", the
> > > same way you can install wget with "apt-get install wget"?
> > >
> > > Similarly, we could help out Mac developers by submitting a
> > > Homebrew<http://brew.sh/> formula
> > > for nutch, and Windows developers would enjoy a
> > > Chocolatey<http://chocolatey.org/> package.
> > > The easier the installation process, the more people want to use nutch!
> > >
> >
> > --
> > *Lewis*
> >
>
--
*Lewis*
Re: Automating nutch installation
Posted by Andrew Pennebaker <ap...@42six.com>.
I've built a .deb for Digital Paintball a few years ago (
http://downloads.yellosoft.us/apps/digitalpaintball-debian-src.tar.gz,
http://downloads.yellosoft.us/apps/digitalpaintball_2.0-19_all.deb).
There's a tutorial for doing this:
https://wiki.debian.org/IntroDebianPackaging
However, I'm using nutch for work, and may have an awkward time charging
hours for contributing to nutch. If we could work together to get a .deb,
maybe even a PPA repo for Debian/Ubuntu, a Homebrew formula for Mac, and a
Chocolatey package for Windows, that would make my life, and probably
thousands of others' lives, a whole lot easier. I'll see what I can get
done during downtime, probably prioritizing Homebrew first.
On Fri, Aug 16, 2013 at 12:06 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Andrew,
> Have you written and maintained Debian packages before?
> I opened a Jira issue for this a whiel back and as far as I know it is a
> far from trivial process but I would be very very interested to get a
> Debiran package for Nutch and look at a mac package if so required.
> There is an issue of who is going to maintain these packages?
> Thanks
> lewis
>
> On Friday, August 16, 2013, Andrew Pennebaker <ap...@42six.com>
> wrote:
> > I think it would make nutch installation easier if we released
> standardized
> > packages.
> >
> > For example, the Nutch tutorial calls for manually downloading a ZIP,
> > extracting the contents, installing ant, compiling nutch with ant, and
> > adjusting PATH.
> >
> > What if instead, Ubuntu users could simply "apt-get install nutch", the
> > same way you can install wget with "apt-get install wget"?
> >
> > Similarly, we could help out Mac developers by submitting a
> > Homebrew<http://brew.sh/> formula
> > for nutch, and Windows developers would enjoy a
> > Chocolatey<http://chocolatey.org/> package.
> > The easier the installation process, the more people want to use nutch!
> >
>
> --
> *Lewis*
>
Re: Automating nutch installation
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Andrew,
Have you written and maintained Debian packages before?
I opened a Jira issue for this a whiel back and as far as I know it is a
far from trivial process but I would be very very interested to get a
Debiran package for Nutch and look at a mac package if so required.
There is an issue of who is going to maintain these packages?
Thanks
lewis
On Friday, August 16, 2013, Andrew Pennebaker <ap...@42six.com> wrote:
> I think it would make nutch installation easier if we released
standardized
> packages.
>
> For example, the Nutch tutorial calls for manually downloading a ZIP,
> extracting the contents, installing ant, compiling nutch with ant, and
> adjusting PATH.
>
> What if instead, Ubuntu users could simply "apt-get install nutch", the
> same way you can install wget with "apt-get install wget"?
>
> Similarly, we could help out Mac developers by submitting a
> Homebrew<http://brew.sh/> formula
> for nutch, and Windows developers would enjoy a
> Chocolatey<http://chocolatey.org/> package.
> The easier the installation process, the more people want to use nutch!
>
--
*Lewis*