You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2010/01/14 12:05:41 UTC

Ivy snapshot related nit

So if Hadoop Core puts up a new snapshot, it looks like Ivy pulls it down but does not
delete any older version which may be locally cached, and so the snapshots get mixed
up on the classpath.

apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn up
At revision 899155.
apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn status
apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf build
apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant && ant test
[...]
BUILD SUCCESSFUL
[...]
test-core:
    [mkdir] Created dir: /home/apurtell/src/Hadoop/hbase-trunk/build/test/logs
    [junit] Running org.apache.hadoop.hbase.TestCompare
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.086 sec
    [junit] Running org.apache.hadoop.hbase.TestEmptyMetaInfo
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.251 sec
    [junit] Test org.apache.hadoop.hbase.TestEmptyMetaInfo FAILED

Tail of build/test/TEST-org.apache.hadoop.hbase.TestEmptyMetaInfo.txt:

Testcase: testEmptyMetaInfo took 2.237 sec
        Caused an ERROR
org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
java.lang.NoSuchMethodError: org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
        at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:283)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:233)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1416)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1371)
        at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:428)
        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:292)
        at org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:124)

Blowing away the Ivy cache resolved this.

apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf ~/.ivy2/cache
apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant && ant test
[...]

And all is well. 

Hitting this for the first time at 3am is annoying. 

Any chance there is some update to the Ivy config which can handle this 
automagically? 

   - Andy

Re: Ivy snapshot related nit

Posted by Kay Kay <ka...@gmail.com>.
On 1/15/10 10:25 AM, Andrew Purtell wrote:
>> Been looking at IVY-938 - that seems to be an issue
>> with Ivy in recognizing this .
>>      
> That's good enough for me. Eventually it will be fixed then I presume.
>
> I'll add a note to the wiki.
>    
My bad !! That was a configuration issue - HBASE-2134 should be in that 
fixes the issue.

>     - Andy
>
>
> ----- Original Message ----
>    
>> From: Kay Kay<ka...@gmail.com>
>> To: hbase-dev@hadoop.apache.org
>> Sent: Thu, January 14, 2010 7:14:07 PM
>> Subject: Re: Ivy snapshot related nit
>>
>> On 1/14/10 3:05 AM, Andrew Purtell wrote:
>>      
>>> So if Hadoop Core puts up a new snapshot, it looks like Ivy pulls it down but
>>>        
>> does not
>>      
>>> delete any older version which may be locally cached, and so the snapshots get
>>>        
>> mixed
>>      
>>> up on the classpath.
>>>
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn up
>>> At revision 899155.
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn status
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf build
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&   ant test
>>> [...]
>>> BUILD SUCCESSFUL
>>> [...]
>>> test-core:
>>>       [mkdir] Created dir:
>>>        
>> /home/apurtell/src/Hadoop/hbase-trunk/build/test/logs
>>      
>>>       [junit] Running org.apache.hadoop.hbase.TestCompare
>>>       [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.086 sec
>>>       [junit] Running org.apache.hadoop.hbase.TestEmptyMetaInfo
>>>       [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.251 sec
>>>       [junit] Test org.apache.hadoop.hbase.TestEmptyMetaInfo FAILED
>>>
>>> Tail of build/test/TEST-org.apache.hadoop.hbase.TestEmptyMetaInfo.txt:
>>>
>>> Testcase: testEmptyMetaInfo took 2.237 sec
>>>           Caused an ERROR
>>>
>>>        
>> org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
>>      
>>> java.lang.NoSuchMethodError:
>>>        
>> org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:283)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:233)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1416)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1371)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:428)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:292)
>>      
>>>           at
>>>        
>> org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:124)
>>      
>>> Blowing away the Ivy cache resolved this.
>>>
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf ~/.ivy2/cache
>>> apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&   ant test
>>> [...]
>>>
>>> And all is well.
>>>
>>> Hitting this for the first time at 3am is annoying.
>>>
>>> Any chance there is some update to the Ivy config which can handle this
>>> automagically?
>>>
>>>
>>>        
>> Sorry about that.. Been looking at IVY-938 - that seems to be an issue
>> with Ivy in recognizing this . Can you log an improvement if this needs
>> to be tracked .
>>
>> An optimization for the ivy cache deletion though.  You can do  $ rm -rf
>> ~/.ivy2/cache/org.apache.hadoop/* as opposed to everything else in the
>> cache.
>>
>>      
>>>      - Andy
>>>
>>>
>>>
>>>
>>>
>>>
>>>        
>
>
>
>
>
>

Re: Ivy snapshot related nit

Posted by Andrew Purtell <ap...@apache.org>.
> Been looking at IVY-938 - that seems to be an issue 
> with Ivy in recognizing this .

That's good enough for me. Eventually it will be fixed then I presume. 

I'll add a note to the wiki. 

   - Andy


----- Original Message ----
> From: Kay Kay <ka...@gmail.com>
> To: hbase-dev@hadoop.apache.org
> Sent: Thu, January 14, 2010 7:14:07 PM
> Subject: Re: Ivy snapshot related nit
> 
> On 1/14/10 3:05 AM, Andrew Purtell wrote:
> > So if Hadoop Core puts up a new snapshot, it looks like Ivy pulls it down but 
> does not
> > delete any older version which may be locally cached, and so the snapshots get 
> mixed
> > up on the classpath.
> >
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn up
> > At revision 899155.
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn status
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf build
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&  ant test
> > [...]
> > BUILD SUCCESSFUL
> > [...]
> > test-core:
> >      [mkdir] Created dir: 
> /home/apurtell/src/Hadoop/hbase-trunk/build/test/logs
> >      [junit] Running org.apache.hadoop.hbase.TestCompare
> >      [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.086 sec
> >      [junit] Running org.apache.hadoop.hbase.TestEmptyMetaInfo
> >      [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.251 sec
> >      [junit] Test org.apache.hadoop.hbase.TestEmptyMetaInfo FAILED
> >
> > Tail of build/test/TEST-org.apache.hadoop.hbase.TestEmptyMetaInfo.txt:
> >
> > Testcase: testEmptyMetaInfo took 2.237 sec
> >          Caused an ERROR
> > 
> org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
> > java.lang.NoSuchMethodError: 
> org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
> >          at 
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:283)
> >          at 
> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:233)
> >          at 
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1416)
> >          at 
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1371)
> >          at 
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:428)
> >          at 
> org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:292)
> >          at 
> org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:124)
> >
> > Blowing away the Ivy cache resolved this.
> >
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf ~/.ivy2/cache
> > apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&  ant test
> > [...]
> >
> > And all is well.
> >
> > Hitting this for the first time at 3am is annoying.
> >
> > Any chance there is some update to the Ivy config which can handle this
> > automagically?
> >
> >    
> Sorry about that.. Been looking at IVY-938 - that seems to be an issue 
> with Ivy in recognizing this . Can you log an improvement if this needs 
> to be tracked .
> 
> An optimization for the ivy cache deletion though.  You can do  $ rm -rf 
> ~/.ivy2/cache/org.apache.hadoop/* as opposed to everything else in the 
> cache.
> 
> >     - Andy
> >
> >
> >
> >
> >
> >

Re: Ivy snapshot related nit

Posted by Kay Kay <ka...@gmail.com>.
On 1/14/10 3:05 AM, Andrew Purtell wrote:
> So if Hadoop Core puts up a new snapshot, it looks like Ivy pulls it down but does not
> delete any older version which may be locally cached, and so the snapshots get mixed
> up on the classpath.
>
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn up
> At revision 899155.
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ svn status
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf build
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&  ant test
> [...]
> BUILD SUCCESSFUL
> [...]
> test-core:
>      [mkdir] Created dir: /home/apurtell/src/Hadoop/hbase-trunk/build/test/logs
>      [junit] Running org.apache.hadoop.hbase.TestCompare
>      [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.086 sec
>      [junit] Running org.apache.hadoop.hbase.TestEmptyMetaInfo
>      [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.251 sec
>      [junit] Test org.apache.hadoop.hbase.TestEmptyMetaInfo FAILED
>
> Tail of build/test/TEST-org.apache.hadoop.hbase.TestEmptyMetaInfo.txt:
>
> Testcase: testEmptyMetaInfo took 2.237 sec
>          Caused an ERROR
> org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
> java.lang.NoSuchMethodError: org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/ipc/VersionedProtocol;
>          at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:283)
>          at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:233)
>          at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1416)
>          at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1371)
>          at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:428)
>          at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:292)
>          at org.apache.hadoop.hbase.HBaseClusterTestCase.setUp(HBaseClusterTestCase.java:124)
>
> Blowing away the Ivy cache resolved this.
>
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ rm -rf ~/.ivy2/cache
> apurtell@latitude:~/src/Hadoop/hbase-trunk$ ant&&  ant test
> [...]
>
> And all is well.
>
> Hitting this for the first time at 3am is annoying.
>
> Any chance there is some update to the Ivy config which can handle this
> automagically?
>
>    
Sorry about that.. Been looking at IVY-938 - that seems to be an issue 
with Ivy in recognizing this . Can you log an improvement if this needs 
to be tracked .

An optimization for the ivy cache deletion though.  You can do  $ rm -rf 
~/.ivy2/cache/org.apache.hadoop/* as opposed to everything else in the 
cache.

>     - Andy
>
>
>
>
>
>