You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2011/12/15 13:59:09 UTC
Modifying Nutch Ivy & Maven settings [WAS] Re: [jira] [Created]
(NUTCH-1225) Migrate CrawlDBScanner to MapReduce API
Hi Markus,
I thought I would branch off from your thread here as I see this as a
different problem (albeit substantially more minor in nature). The
question were trying to address here is
> Does anyone know how i can modify Ivy to use Apache's maven repo for the
> Hadoop dependencies? It keeps trying to load it from maven central where the
> 0.21 pom is not present.
Do you mean repository.apache.org. as oppose to http://repo1.maven.org/maven2 ?
If you look in ivy/ivysettings.xml you will see where we are currently
pulling dependencies from. Can anyone confirm this is the case?
-----------------------------------------------------
Further to this, on a slightly different topic, between lines 28-31 in
build.xml we can see
<!-- define Maven coordinates, repository url and artifacts name etc -->
<property name="groupId" value="org.apache.nutch" />
<property name="artifactId" value="nutch" />
<property name="maven-repository-url"
value="https://repository.apache.org/service/local/staging/deploy/maven2"
/>
The ${maven-repository-url} doesn't resolve to anywhere and returns an
HTTP 400ish response. Any thoughts on this one!
Sorry for going off at a tangent here guys!
Re: Modifying Nutch Ivy & Maven settings [WAS] Re: [jira] [Created] (NUTCH-1225) Migrate CrawlDBScanner to MapReduce API
Posted by Markus Jelsma <ma...@openindex.io>.
I manually added 0.21 and proceed with porting. I now at least have both API's
and mapfileoutputformat.
On Thursday 15 December 2011 13:59:09 Lewis John Mcgibbney wrote:
> Hi Markus,
>
> I thought I would branch off from your thread here as I see this as a
> different problem (albeit substantially more minor in nature). The
> question were trying to address here is
>
> > Does anyone know how i can modify Ivy to use Apache's maven repo for the
> > Hadoop dependencies? It keeps trying to load it from maven central where
> > the 0.21 pom is not present.
>
> Do you mean repository.apache.org. as oppose to
> http://repo1.maven.org/maven2 ?
>
> If you look in ivy/ivysettings.xml you will see where we are currently
> pulling dependencies from. Can anyone confirm this is the case?
> -----------------------------------------------------
> Further to this, on a slightly different topic, between lines 28-31 in
> build.xml we can see
>
> <!-- define Maven coordinates, repository url and artifacts name etc -->
> <property name="groupId" value="org.apache.nutch" />
> <property name="artifactId" value="nutch" />
> <property name="maven-repository-url"
> value="https://repository.apache.org/service/local/staging/deploy/maven2"
> />
>
> The ${maven-repository-url} doesn't resolve to anywhere and returns an
> HTTP 400ish response. Any thoughts on this one!
>
> Sorry for going off at a tangent here guys!
--
Markus Jelsma - CTO - Openindex
Re: Modifying Nutch Ivy & Maven settings [WAS] Re: [jira] [Created]
(NUTCH-1225) Migrate CrawlDBScanner to MapReduce API
Posted by Lewis John Mcgibbney <le...@gmail.com>.
>>
>> Do you mean repository.apache.org. as oppose to
>> http://repo1.maven.org/maven2 ?
>
> Yes. It a bit outdated in the settings. Doesn't matter, i cannot seem to tell
> Ivy to load the poms for a specific dep from a specific repo. Ivy confuses me
> ;)
ivy/ivysettings.xml must be the correct file to specify from where and
how we pull the dependencies. I've also tried editing this is the past
(with Nutchgora) and didn't get anywhere. I wonder if you have to
flush your local .ivy repository then respecify where you wish to pull
them from. If you look at lines 73-79, we specifically attempt to
resolve all artifacts to be bult locally.
<!--
This forces a requirement for other nutch-artifacts to be built locally
rather than look for them online.
-->
<module organisation="org.apache.nutch" name=".*" resolver="internal"/>
</modules>
</ivysettings>
>
> This is not for pulling depecencies if im not mistaken.
>
Correct, but it still points no nowhere!!!