You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Rodrigo Schmidt <ro...@facebook.com> on 2010/03/18 07:02:44 UTC

Building offline

How do I disable the ivy download to run unit tests? I need to replace hadoop-core.jar for one I created with some extra logging to debug a unit test for mapreduce.

Right now I’m just erasing some lines from build.xml, but I’m quite sure there is an easier way to do that.

Thanks,
Rodrigo

Re: Building offline

Posted by Karthik K <os...@gmail.com>.

To add to chaitanya's suggestions earlier - you can set the version number
in libraries.properties / ivy.xml to be something else (for the patched
version you are working on, say 0.22.1 or 0.23.0 ) and publish using the
'ant mvn-install'  task.

Then in mapreduce's ivy.xml - just change the libraries.properties to the
new version that you had put and build ,to include your new version. Let us
know how it goes ( no need to go , --offline etc. ).




On Thu, Mar 18, 2010 at 7:54 PM, Rodrigo Schmidt <ro...@facebook.com>wrote:

>
> Ant clean is not erasing ~/.ivy2/cache, so I have to explicitly remove the
> files if I want to replace the jar for a newly compiled one. This doesn't
> sound right, though.
>
> Cheers,
> Rodrigo
>
> On 3/18/10 12:54 AM, "chaitanya krishna" <ch...@gmail.com>
> wrote:
>
> > Hi Rodrigo,
> >
> >   AFAIK,  I dont find any other way of cross-project debugging as of now.
> > Also, -Doffline=true is not working (a bug). Once this works, I think we
> can
> > work around by putting hadoop-core.jar in
> > mapred-trunk/build/ivy/lib/Hadoop/common (not sure though).
> >
> >  Also, I think doing an ant clean before running ant test
> > -Dresolvers=internal might be the actual solution instead of deleting
> > hadoop-core.jar in ~/.ivy2/cache, because the cached jar might be the one
> in
> > build/ivy/lib/Hadoop/common.
> >
> > Regards,
> > Chaitanya.
> >
> > On Thu, Mar 18, 2010 at 1:01 PM, Rodrigo Schmidt <rodrigo@facebook.com
> >wrote:
> >
> >>
> >> Thanks Chaitanya!
> >>
> >> Didn't work at first. I tried many times in different ways, connected to
> >> the
> >> Internet and not.
> >>
> >> When I was about to give up I realized ivy might be caching some stuff,
> >> which turned out to be true. I had to delete the hadoop-core.jar file
> >> inside
> >> ~/ivy2/cache for it to work.
> >>
> >> This whole ivy2 and maven dependencies look too complicated for offline
> use
> >> and cross-project debugging. Isn't there an easier way to do things?
> >>
> >> Cheers,
> >> Rodrigo
> >>
> >>
> >> On 3/17/10 11:27 PM, "chaitanya krishna" <ch...@gmail.com>
> >> wrote:
> >>
> >>> In case you want to use hadoop-core.jar containing your modifications
> in
> >>> common, i think the following works:
> >>>
> >>> 1. ant mvn-install in common project. This will create hadoop-core.jar
> >> and
> >>> installs it in the local m2 repository.
> >>> 2. ant test -Dresolvers=internal in mapreduce project. This will make
> >> sure
> >>> that the hadoop-core.jar is taken from local m2 repository.
> >>>
> >>> Hope this helps.
> >>>
> >>> -Chaitanya.
> >>>
> >>> On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <
> rodrigo@facebook.com
> >>> wrote:
> >>>
> >>>>
> >>>> I tried that but my logs still didn't show up. To test if it really
> >> worked,
> >>>> I disabled my Internet connection and tried it, which failed. Did I
> miss
> >>>> anything?
> >>>>
> >>>>
> >>>> Output below:
> >>>>
> >>>>
> >>>> $ant -Doffline=yes test
> >>>> Buildfile: build.xml
> >>>>
> >>>> ivy-download:
> >>>>
> >>>> ivy-init-dirs:
> >>>>
> >>>> ivy-probe-antlib:
> >>>>
> >>>> ivy-init-antlib:
> >>>>
> >>>> ivy-init:
> >>>> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
> >>>> http://ant.apache.org/ivy/ ::
> >>>> :: loading settings :: file =
> >>>> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
> >>>>
> >>>> ivy-resolve-common:
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve] You probably access the destination server through a
> proxy
> >>>> server that is not well configured.
> >>>> [ivy:resolve]
> >>>> [ivy:resolve] :: problems summary ::
> >>>> [ivy:resolve] :::: WARNINGS
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>>
> >>
> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
> >>>>
> >>
> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0
> >>>> -SNAPSHOT/maven-metadata.xml>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> >> https://re
> >>>>
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> >> https://re
> >>>>
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> >>>> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/org/apache/
> >>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> >>>> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/org/apache/
> >>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]         module not found:
> >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
> >>>> [ivy:resolve]     ==== apache-snapshot: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> >> https://reposi
> >>>>
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
> >>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> >> https://reposi
> >>>>
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
> >>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     ==== maven2: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> >>>> hadoop-core-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/org/apache/hado
> >>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> >>>> hadoop-core-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/org/apache/hado
> >>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<
> >> https://repository
> >>>> .
> >>
> apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t
> >>>> est/0.22.0-SNAPSHOT/maven-metadata.xml>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>>
> >>>>
> >>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
> >>>>
> >>
> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>>
> >>>>
> >>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
> >>>>
> >>
> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> >>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/o
> >>>>
> >>
> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
> >>>> SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> >>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/o
> >>>>
> >>
> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
> >>>> SNAPSHOT.jar>
> >>>> [ivy:resolve]         module not found:
> >>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
> >>>> [ivy:resolve]     ==== apache-snapshot: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>>
> >>
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http
> >>>> s://
> >>
> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
> >>>>
> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>>
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>>
> >>
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http
> >>>> s://
> >>
> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
> >>>>
> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     ==== maven2: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> >>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/org/a
> >>>>
> >>
> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
> >>>> SHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>>
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> >>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/org/a
> >>>>
> >>
> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
> >>>> SHOT.jar>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>>
> >>
> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
> >>>>
> >>
> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0
> >>>> -SNAPSHOT/maven-metadata.xml>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> >> https://re
> >>>>
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repository.apache.org not found.
> >>>> url=
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> >> https://re
> >>>>
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> >>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/org/apache/
> >>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]     Host repo1.maven.org not found.
> >>>> url=
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> >>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/org/apache/
> >>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]         module not found:
> >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
> >>>> [ivy:resolve]     ==== apache-snapshot: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> >> https://reposi
> >>>>
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
> >>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> >> https://reposi
> >>>>
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
> >>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]     ==== maven2: tried
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> >>>> hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> >> http://repo1.maven.org/maven2/org/apache/hado
> >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >>>> [ivy:resolve]       -- artifact
> >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> >>>> [ivy:resolve]
> >>>>
> >>>>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> >>>> hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> >> http://repo1.maven.org/maven2/org/apache/hado
> >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >>>> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
> >>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >>>> [ivy:resolve]         ::
> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT:
> >> not
> >>>> found
> >>>> [ivy:resolve]         ::
> >>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
> >>>> not found
> >>>> [ivy:resolve]         ::
> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
> >> not
> >>>> found
> >>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >>>> [ivy:resolve]
> >>>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> >>>>
> >>>>
> >>>>
> >>>> On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:
> >>>>
> >>>>> Use -Doffline=yes switch while running the tests.
> >>>>> Amar
> >>>>>
> >>>>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> How do I disable the ivy download to run unit tests? I need to
> replace
> >>>>> hadoop-core.jar for one I created with some extra logging to debug a
> >> unit
> >>>> test
> >>>>> for mapreduce.
> >>>>>
> >>>>> Right now I'm just erasing some lines from build.xml, but I'm quite
> >> sure
> >>>> there
> >>>>> is an easier way to do that.
> >>>>>
> >>>>> Thanks,
> >>>>> Rodrigo
> >>>>>
> >>>>
> >>>>
> >>
> >>
>
>

Re: Building offline

Posted by Rodrigo Schmidt <ro...@facebook.com>.

Ant clean is not erasing ~/.ivy2/cache, so I have to explicitly remove the
files if I want to replace the jar for a newly compiled one. This doesn't
sound right, though.

Cheers,
Rodrigo

On 3/18/10 12:54 AM, "chaitanya krishna" <ch...@gmail.com>
wrote:

> Hi Rodrigo,
>
>   AFAIK,  I dont find any other way of cross-project debugging as of now.
> Also, -Doffline=true is not working (a bug). Once this works, I think we can
> work around by putting hadoop-core.jar in
> mapred-trunk/build/ivy/lib/Hadoop/common (not sure though).
>
>  Also, I think doing an ant clean before running ant test
> -Dresolvers=internal might be the actual solution instead of deleting
> hadoop-core.jar in ~/.ivy2/cache, because the cached jar might be the one in
> build/ivy/lib/Hadoop/common.
>
> Regards,
> Chaitanya.
>
> On Thu, Mar 18, 2010 at 1:01 PM, Rodrigo Schmidt <ro...@facebook.com>wrote:
>
>>
>> Thanks Chaitanya!
>>
>> Didn't work at first. I tried many times in different ways, connected to
>> the
>> Internet and not.
>>
>> When I was about to give up I realized ivy might be caching some stuff,
>> which turned out to be true. I had to delete the hadoop-core.jar file
>> inside
>> ~/ivy2/cache for it to work.
>>
>> This whole ivy2 and maven dependencies look too complicated for offline use
>> and cross-project debugging. Isn't there an easier way to do things?
>>
>> Cheers,
>> Rodrigo
>>
>>
>> On 3/17/10 11:27 PM, "chaitanya krishna" <ch...@gmail.com>
>> wrote:
>>
>>> In case you want to use hadoop-core.jar containing your modifications in
>>> common, i think the following works:
>>>
>>> 1. ant mvn-install in common project. This will create hadoop-core.jar
>> and
>>> installs it in the local m2 repository.
>>> 2. ant test -Dresolvers=internal in mapreduce project. This will make
>> sure
>>> that the hadoop-core.jar is taken from local m2 repository.
>>>
>>> Hope this helps.
>>>
>>> -Chaitanya.
>>>
>>> On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <rodrigo@facebook.com
>>> wrote:
>>>
>>>>
>>>> I tried that but my logs still didn't show up. To test if it really
>> worked,
>>>> I disabled my Internet connection and tried it, which failed. Did I miss
>>>> anything?
>>>>
>>>>
>>>> Output below:
>>>>
>>>>
>>>> $ant -Doffline=yes test
>>>> Buildfile: build.xml
>>>>
>>>> ivy-download:
>>>>
>>>> ivy-init-dirs:
>>>>
>>>> ivy-probe-antlib:
>>>>
>>>> ivy-init-antlib:
>>>>
>>>> ivy-init:
>>>> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
>>>> http://ant.apache.org/ivy/ ::
>>>> :: loading settings :: file =
>>>> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
>>>>
>>>> ivy-resolve-common:
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve] You probably access the destination server through a proxy
>>>> server that is not well configured.
>>>> [ivy:resolve]
>>>> [ivy:resolve] :: problems summary ::
>>>> [ivy:resolve] :::: WARNINGS
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>>
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>>>>
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0
>>>> -SNAPSHOT/maven-metadata.xml>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
>> https://re
>>>>
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
>> https://re
>>>>
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>>>> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/org/apache/
>>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>>>> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/org/apache/
>>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]         module not found:
>>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
>>>> [ivy:resolve]     ==== apache-snapshot: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
>> https://reposi
>>>>
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
>> https://reposi
>>>>
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     ==== maven2: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>>>> hadoop-core-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/org/apache/hado
>>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>>>> hadoop-core-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/org/apache/hado
>>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<
>> https://repository
>>>> .
>> apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t
>>>> est/0.22.0-SNAPSHOT/maven-metadata.xml>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>>
>>>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>>>>
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>>
>>>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>>>>
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/o
>>>>
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>>>> SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/o
>>>>
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>>>> SNAPSHOT.jar>
>>>> [ivy:resolve]         module not found:
>>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
>>>> [ivy:resolve]     ==== apache-snapshot: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>>
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http
>>>> s://
>> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>>>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>>
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http
>>>> s://
>> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>>>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     ==== maven2: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/org/a
>>>>
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>>>> SHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/org/a
>>>>
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>>>> SHOT.jar>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>>
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>>>>
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0
>>>> -SNAPSHOT/maven-metadata.xml>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
>> https://re
>>>>
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repository.apache.org not found.
>>>> url=
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
>> https://re
>>>>
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/org/apache/
>>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]     Host repo1.maven.org not found.
>>>> url=
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/org/apache/
>>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]         module not found:
>>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>>>> [ivy:resolve]     ==== apache-snapshot: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
>> https://reposi
>>>>
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
>> https://reposi
>>>>
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]     ==== maven2: tried
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>>>> hadoop-hdfs-0.22.0-SNAPSHOT.pom<
>> http://repo1.maven.org/maven2/org/apache/hado
>>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
>>>> [ivy:resolve]       -- artifact
>>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>>>> [ivy:resolve]
>>>>
>>>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>>>> hadoop-hdfs-0.22.0-SNAPSHOT.jar<
>> http://repo1.maven.org/maven2/org/apache/hado
>>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
>>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
>>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT:
>> not
>>>> found
>>>> [ivy:resolve]         ::
>>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
>>>> not found
>>>> [ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
>> not
>>>> found
>>>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [ivy:resolve]
>>>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>>>
>>>>
>>>>
>>>> On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:
>>>>
>>>>> Use -Doffline=yes switch while running the tests.
>>>>> Amar
>>>>>
>>>>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> How do I disable the ivy download to run unit tests? I need to replace
>>>>> hadoop-core.jar for one I created with some extra logging to debug a
>> unit
>>>> test
>>>>> for mapreduce.
>>>>>
>>>>> Right now I'm just erasing some lines from build.xml, but I'm quite
>> sure
>>>> there
>>>>> is an easier way to do that.
>>>>>
>>>>> Thanks,
>>>>> Rodrigo
>>>>>
>>>>
>>>>
>>
>>

Re: Building offline

Posted by chaitanya krishna <ch...@gmail.com>.

Hi Rodrigo,

  AFAIK,  I dont find any other way of cross-project debugging as of now.
Also, -Doffline=true is not working (a bug). Once this works, I think we can
work around by putting hadoop-core.jar in
mapred-trunk/build/ivy/lib/Hadoop/common (not sure though).

 Also, I think doing an ant clean before running ant test
-Dresolvers=internal might be the actual solution instead of deleting
hadoop-core.jar in ~/.ivy2/cache, because the cached jar might be the one in
build/ivy/lib/Hadoop/common.

Regards,
Chaitanya.

On Thu, Mar 18, 2010 at 1:01 PM, Rodrigo Schmidt <ro...@facebook.com>wrote:

>
> Thanks Chaitanya!
>
> Didn't work at first. I tried many times in different ways, connected to
> the
> Internet and not.
>
> When I was about to give up I realized ivy might be caching some stuff,
> which turned out to be true. I had to delete the hadoop-core.jar file
> inside
> ~/ivy2/cache for it to work.
>
> This whole ivy2 and maven dependencies look too complicated for offline use
> and cross-project debugging. Isn't there an easier way to do things?
>
> Cheers,
> Rodrigo
>
>
> On 3/17/10 11:27 PM, "chaitanya krishna" <ch...@gmail.com>
> wrote:
>
> > In case you want to use hadoop-core.jar containing your modifications in
> > common, i think the following works:
> >
> > 1. ant mvn-install in common project. This will create hadoop-core.jar
> and
> > installs it in the local m2 repository.
> > 2. ant test -Dresolvers=internal in mapreduce project. This will make
> sure
> > that the hadoop-core.jar is taken from local m2 repository.
> >
> > Hope this helps.
> >
> > -Chaitanya.
> >
> > On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <rodrigo@facebook.com
> >wrote:
> >
> >>
> >> I tried that but my logs still didn't show up. To test if it really
> worked,
> >> I disabled my Internet connection and tried it, which failed. Did I miss
> >> anything?
> >>
> >>
> >> Output below:
> >>
> >>
> >> $ant -Doffline=yes test
> >> Buildfile: build.xml
> >>
> >> ivy-download:
> >>
> >> ivy-init-dirs:
> >>
> >> ivy-probe-antlib:
> >>
> >> ivy-init-antlib:
> >>
> >> ivy-init:
> >> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
> >> http://ant.apache.org/ivy/ ::
> >> :: loading settings :: file =
> >> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
> >>
> >> ivy-resolve-common:
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve] You probably access the destination server through a proxy
> >> server that is not well configured.
> >> [ivy:resolve]
> >> [ivy:resolve] :: problems summary ::
> >> [ivy:resolve] :::: WARNINGS
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>
> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
> >>
> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0
> >> -SNAPSHOT/maven-metadata.xml>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> https://re
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> https://re
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> >> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/org/apache/
> >> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> >> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/org/apache/
> >> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]         module not found:
> >> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
> >> [ivy:resolve]     ==== apache-snapshot: tried
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<
> https://reposi
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
> >> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<
> https://reposi
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
> >> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     ==== maven2: tried
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> >> hadoop-core-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/org/apache/hado
> >> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> >> hadoop-core-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/org/apache/hado
> >> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<
> https://repository
> >> .
> apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t
> >> est/0.22.0-SNAPSHOT/maven-metadata.xml>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>
> >>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
> >>
> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>
> >>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
> >>
> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> >> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/o
> >>
> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
> >> SNAPSHOT.pom>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> >> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/o
> >>
> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
> >> SNAPSHOT.jar>
> >> [ivy:resolve]         module not found:
> >> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
> >> [ivy:resolve]     ==== apache-snapshot: tried
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http
> >> s://
> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
> >> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >>
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http
> >> s://
> repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
> >> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     ==== maven2: tried
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> >> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/org/a
> >>
> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
> >> SHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> >> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/org/a
> >>
> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
> >> SHOT.jar>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >>
> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
> >>
> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0
> >> -SNAPSHOT/maven-metadata.xml>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> https://re
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]     Host repository.apache.org not found.
> >> url=
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> >> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> https://re
> >>
> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
> >> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> >> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/org/apache/
> >> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]     Host repo1.maven.org not found.
> >> url=
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> >> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/org/apache/
> >> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]         module not found:
> >> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
> >> [ivy:resolve]     ==== apache-snapshot: tried
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> https://reposi
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
> >> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> >> [ivy:resolve]
> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> https://reposi
> >>
> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
> >> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]     ==== maven2: tried
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> >> hadoop-hdfs-0.22.0-SNAPSHOT.pom<
> http://repo1.maven.org/maven2/org/apache/hado
> >> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
> >> [ivy:resolve]       -- artifact
> >> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> >> [ivy:resolve]
> >>
> >>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> >> hadoop-hdfs-0.22.0-SNAPSHOT.jar<
> http://repo1.maven.org/maven2/org/apache/hado
> >> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
> >> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
> >> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >> [ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT:
> not
> >> found
> >> [ivy:resolve]         ::
> >> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
> >> not found
> >> [ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
> not
> >> found
> >> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> >> [ivy:resolve]
> >> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> >>
> >>
> >>
> >> On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:
> >>
> >>> Use -Doffline=yes switch while running the tests.
> >>> Amar
> >>>
> >>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
> >>>
> >>>
> >>>
> >>> How do I disable the ivy download to run unit tests? I need to replace
> >>> hadoop-core.jar for one I created with some extra logging to debug a
> unit
> >> test
> >>> for mapreduce.
> >>>
> >>> Right now I'm just erasing some lines from build.xml, but I'm quite
> sure
> >> there
> >>> is an easier way to do that.
> >>>
> >>> Thanks,
> >>> Rodrigo
> >>>
> >>
> >>
>
>

Re: Building offline

Posted by Rodrigo Schmidt <ro...@facebook.com>.

Thanks Chaitanya!

Didn't work at first. I tried many times in different ways, connected to the
Internet and not.

When I was about to give up I realized ivy might be caching some stuff,
which turned out to be true. I had to delete the hadoop-core.jar file inside
~/ivy2/cache for it to work.

This whole ivy2 and maven dependencies look too complicated for offline use
and cross-project debugging. Isn't there an easier way to do things?

Cheers,
Rodrigo


On 3/17/10 11:27 PM, "chaitanya krishna" <ch...@gmail.com>
wrote:

> In case you want to use hadoop-core.jar containing your modifications in
> common, i think the following works:
>
> 1. ant mvn-install in common project. This will create hadoop-core.jar and
> installs it in the local m2 repository.
> 2. ant test -Dresolvers=internal in mapreduce project. This will make sure
> that the hadoop-core.jar is taken from local m2 repository.
>
> Hope this helps.
>
> -Chaitanya.
>
> On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <ro...@facebook.com>wrote:
>
>>
>> I tried that but my logs still didn't show up. To test if it really worked,
>> I disabled my Internet connection and tried it, which failed. Did I miss
>> anything?
>>
>>
>> Output below:
>>
>>
>> $ant -Doffline=yes test
>> Buildfile: build.xml
>>
>> ivy-download:
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
>> http://ant.apache.org/ivy/ ::
>> :: loading settings :: file =
>> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0
>> -SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>> hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>> hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository
>> .apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t
>> est/0.22.0-SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/o
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>> SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/o
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>> SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http
>> s://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http
>> s://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/a
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>> SHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/a
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>> SHOT.jar>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0
>> -SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>> hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>> hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT: not
>> found
>> [ivy:resolve]         ::
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
>> not found
>> [ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not
>> found
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>>
>>
>> On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:
>>
>>> Use -Doffline=yes switch while running the tests.
>>> Amar
>>>
>>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
>>>
>>>
>>>
>>> How do I disable the ivy download to run unit tests? I need to replace
>>> hadoop-core.jar for one I created with some extra logging to debug a unit
>> test
>>> for mapreduce.
>>>
>>> Right now I'm just erasing some lines from build.xml, but I'm quite sure
>> there
>>> is an easier way to do that.
>>>
>>> Thanks,
>>> Rodrigo
>>>
>>
>>

Re: Building offline

Posted by chaitanya krishna <ch...@gmail.com>.

In case you want to use hadoop-core.jar containing your modifications in
common, i think the following works:

1. ant mvn-install in common project. This will create hadoop-core.jar and
installs it in the local m2 repository.
2. ant test -Dresolvers=internal in mapreduce project. This will make sure
that the hadoop-core.jar is taken from local m2 repository.

Hope this helps.

-Chaitanya.

On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <ro...@facebook.com>wrote:

>
> I tried that but my logs still didn't show up. To test if it really worked,
> I disabled my Internet connection and tried it, which failed. Did I miss
> anything?
>
>
> Output below:
>
>
> $ant -Doffline=yes test
> Buildfile: build.xml
>
> ivy-download:
>
> ivy-init-dirs:
>
> ivy-probe-antlib:
>
> ivy-init-antlib:
>
> ivy-init:
> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
> http://ant.apache.org/ivy/ ::
> :: loading settings :: file =
> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
>
> ivy-resolve-common:
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve] You probably access the destination server through a proxy
> server that is not well configured.
> [ivy:resolve]
> [ivy:resolve] :: problems summary ::
> [ivy:resolve] :::: WARNINGS
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]         module not found:
> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
> [ivy:resolve]     ==== apache-snapshot: tried
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     ==== maven2: tried
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
> hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
>
> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]         module not found:
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
> [ivy:resolve]     ==== apache-snapshot: tried
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     ==== maven2: tried
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repository.apache.org not found.
> url=
> https://repository.apache.org/content/repositories/snapshots/org/apache/
> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]     Host repo1.maven.org not found.
> url=
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]         module not found:
> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
> [ivy:resolve]     ==== apache-snapshot: tried
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> [ivy:resolve]
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]     ==== maven2: tried
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
> [ivy:resolve]       -- artifact
> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
> [ivy:resolve]
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
> hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT: not
> found
> [ivy:resolve]         ::
> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
> not found
> [ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not
> found
> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]
> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
>
>
> On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:
>
> > Use -Doffline=yes switch while running the tests.
> > Amar
> >
> > On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
> >
> >
> >
> > How do I disable the ivy download to run unit tests? I need to replace
> > hadoop-core.jar for one I created with some extra logging to debug a unit
> test
> > for mapreduce.
> >
> > Right now I'm just erasing some lines from build.xml, but I'm quite sure
> there
> > is an easier way to do that.
> >
> > Thanks,
> > Rodrigo
> >
>
>

Re: Building offline

Posted by Rodrigo Schmidt <ro...@facebook.com>.

I tried that but my logs still didn't show up. To test if it really worked,
I disabled my Internet connection and tried it, which failed. Did I miss
anything?

Output below:

$ant -Doffline=yes test
Buildfile: build.xml

ivy-download:

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
http://ant.apache.org/ivy/ ::
:: loading settings :: file =
/Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml

ivy-resolve-common:
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] You probably access the destination server through a proxy
server that is not well configured.
[ivy:resolve] 
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
HOT/hadoop-core-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
HOT/hadoop-core-0.22.0-SNAPSHOT.jar
[ivy:resolve]         module not found:
org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
[ivy:resolve]     ==== apache-snapshot: tried
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar
[ivy:resolve]     ==== maven2: tried
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
hadoop-core-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
hadoop-core-0.22.0-SNAPSHOT.jar
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]         module not found:
org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
[ivy:resolve]     ==== apache-snapshot: tried
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]     ==== maven2: tried
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repository.apache.org not found.
url=https://repository.apache.org/content/repositories/snapshots/org/apache/
hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]     Host repo1.maven.org not found.
url=http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]         module not found:
org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
[ivy:resolve]     ==== apache-snapshot: tried
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve]      
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]     ==== maven2: tried
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]       -- artifact
org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve]      
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
[ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT: not
found
[ivy:resolve]         :: org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
not found
[ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not
found
[ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] 
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

On 3/17/10 11:14 PM, "Amar Kamat" <am...@yahoo-inc.com> wrote:

> Use -Doffline=yes switch while running the tests.
> Amar
> 
> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:
> 
> 
> 
> How do I disable the ivy download to run unit tests? I need to replace
> hadoop-core.jar for one I created with some extra logging to debug a unit test
> for mapreduce.
> 
> Right now I'm just erasing some lines from build.xml, but I'm quite sure there
> is an easier way to do that.
> 
> Thanks,
> Rodrigo
>

Re: Building offline

Posted by Amar Kamat <am...@yahoo-inc.com>.

Use -Doffline=yes switch while running the tests.
Amar

On 3/18/10 11:32 AM, "Rodrigo Schmidt" <ro...@facebook.com> wrote:



How do I disable the ivy download to run unit tests? I need to replace hadoop-core.jar for one I created with some extra logging to debug a unit test for mapreduce.

Right now I'm just erasing some lines from build.xml, but I'm quite sure there is an easier way to do that.

Thanks,
Rodrigo