You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Owen O'Malley <om...@apache.org> on 2008/08/30 00:28:10 UTC

git clone of hadoop core?

Has anyone succeeded in using git-svn to clone the Hadoop core  
subversion repository with branches? I tried it and it failed. I got  
the following logs. Any suggestions?

-- Owen

% git svn init -s http://svn.apache.org/repos/asf/hadoop/core
Initialized empty Git repository in /export/hadoop-clone/.git/
Using higher level of URL: http://svn.apache.org/repos/asf/hadoop/core  
=> http://svn.apache.org/repos/asf

% git svn fetch
W: Ignoring error from SVN, path probably does not exist: (160013):  
Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/ 
100':
File not found: revision 100, path '/hadoop/core'

W: Do not be alarmed at the above message git-svn is just searching  
aggressively for old history.
This may take a while on large repositories
Found possible branch point: http://svn.apache.org/repos/asf/lucene/hadoop/trunk 
  => http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
Initializing parent: trunk@614596
W: Ignoring error from SVN, path probably does not exist: (160013):  
Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/ 
101':
File not found: revision 101, path '/lucene/hadoop/trunk'

W: Do not be alarmed at the above message git-svn is just searching  
aggressively for old history.
This may take a while on large repositories
RA layer request failed: REPORT request failed on '/repos/asf/!svn/vcc/ 
default': REPORT of '/repos/asf/!svn/vcc/default': 403 Forbidden (http://svn.apache.org 
) at /home/oom/tools/libexec/git-core//git-svn line 3955

Re: git clone of hadoop core?

Posted by Owen O'Malley <ow...@gmail.com>.
I got the clone of trunk working too. No luck getting the branches  
with -s though... I even tried adding a -r with the revision just  
after hbase was taken out to the fetch, which gets a different error.

-- Owen
>>
>>
>>
>>
>>
>>

Re: git clone of hadoop core?

Posted by Steve Loughran <st...@apache.org>.
Christophe Taton wrote:
> Hi Owen,
> 
> I use: git-svn clone http://svn.apache.org/repos/asf/hadoop/core/trunk
> This is a *very* long operation which will download ~70Mb from the svn.
> 
> Hope this helps!
> 
> Cheers,
> Christophe

There's an EU hosted SVN server too, svn.eu.apache.org; this passes back 
all writes to the main svn server but serves up reads from its 
(synchronized) cache. Its closer geographically for anyone in the EU and 
nearby areas, and should put less load on the central servers than 
grabbing a 70mb fetch, so infrastructure are less likely to block the 
operation.

before doing a 70MB download, ping the sites and see which is closer/faster.

-steve

Re: git clone of hadoop core?

Posted by Christophe Taton <ta...@apache.org>.
Hi Owen,

I use: git-svn clone http://svn.apache.org/repos/asf/hadoop/core/trunk
This is a *very* long operation which will download ~70Mb from the svn.

Hope this helps!

Cheers,
Christophe

On Sat, Aug 30, 2008 at 12:28 AM, Owen O'Malley <om...@apache.org> wrote:

> Has anyone succeeded in using git-svn to clone the Hadoop core subversion
> repository with branches? I tried it and it failed. I got the following
> logs. Any suggestions?
>
> -- Owen
>
> % git svn init -s http://svn.apache.org/repos/asf/hadoop/core
> Initialized empty Git repository in /export/hadoop-clone/.git/
> Using higher level of URL: http://svn.apache.org/repos/asf/hadoop/core =>
> http://svn.apache.org/repos/asf
>
> % git svn fetch
> W: Ignoring error from SVN, path probably does not exist: (160013):
> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/100':
> File not found: revision 100, path '/hadoop/core'
>
> W: Do not be alarmed at the above message git-svn is just searching
> aggressively for old history.
> This may take a while on large repositories
> Found possible branch point:
> http://svn.apache.org/repos/asf/lucene/hadoop/trunk =>
> http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
> Initializing parent: trunk@614596
> W: Ignoring error from SVN, path probably does not exist: (160013):
> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/101':
> File not found: revision 101, path '/lucene/hadoop/trunk'
>
> W: Do not be alarmed at the above message git-svn is just searching
> aggressively for old history.
> This may take a while on large repositories
> RA layer request failed: REPORT request failed on
> '/repos/asf/!svn/vcc/default': REPORT of '/repos/asf/!svn/vcc/default': 403
> Forbidden (http://svn.apache.org) at
> /home/oom/tools/libexec/git-core//git-svn line 3955
>
>

Re: git clone of hadoop core?

Posted by Chris K Wensel <ch...@wensel.net>.
very cool, thanks..

btw, looks like rudimentary clones have already been made (but haven't  
tried cloning any yet).

On Sep 12, 2008, at 10:19 AM, Doug Cutting wrote:

> FYI, Jukka Zitting has been exploring using Git with Apache's SVN  
> over on the infrastructure-dev@a.o mailing list.
>
> http://markmail.org/message/fzzy7nepk7olx5fl
>
> I don't know how his methods compare to Owen's & Pat's.
>
> Doug
>
> Chris K Wensel wrote:
>> Just curious if there has been any progress/changes on this.
>> I'm thinking a git repo of Hadoop would be a handy thing. but i  
>> expect having every random developer attempt to clone the apache  
>> hadoop repo via git-svn would be crushing.
>> any thought on how a mirrored git repo would work?
>> cheers,
>> chris
>> On Sep 9, 2008, at 7:04 AM, Patrick Hunt wrote:
>>> Owen, you could be running into one of the following issues:
>>>
>>> Proxy btw you and the server doesn't support the http REPORT  
>>> command:
>>> http://vsingleton.blogspot.com/2008/04/svn-propfind-request-failed-on.html
>>>
>>> or perhaps the svn http access is configured to not allow the http  
>>> REPORT command:
>>> http://svn.haxx.se/users/archive-2005-01/0971.shtml
>>>
>>> Hope this helps.
>>>
>>> Patrick
>>>
>>> Owen O'Malley wrote:
>>>> Has anyone succeeded in using git-svn to clone the Hadoop core  
>>>> subversion repository with branches? I tried it and it failed. I  
>>>> got the following logs. Any suggestions?
>>>> -- Owen
>>>> % git svn init -s http://svn.apache.org/repos/asf/hadoop/core
>>>> Initialized empty Git repository in /export/hadoop-clone/.git/
>>>> Using higher level of URL: http://svn.apache.org/repos/asf/hadoop/core 
>>>>  => http://svn.apache.org/repos/asf
>>>> % git svn fetch
>>>> W: Ignoring error from SVN, path probably does not exist:  
>>>> (160013): Filesystem has no item: REPORT request failed on '/ 
>>>> repos/asf/!svn/bc/100':
>>>> File not found: revision 100, path '/hadoop/core'
>>>> W: Do not be alarmed at the above message git-svn is just  
>>>> searching aggressively for old history.
>>>> This may take a while on large repositories
>>>> Found possible branch point: http://svn.apache.org/repos/asf/lucene/hadoop/trunk 
>>>>  => http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
>>>> Initializing parent: trunk@614596
>>>> W: Ignoring error from SVN, path probably does not exist:  
>>>> (160013): Filesystem has no item: REPORT request failed on '/ 
>>>> repos/asf/!svn/bc/101':
>>>> File not found: revision 101, path '/lucene/hadoop/trunk'
>>>> W: Do not be alarmed at the above message git-svn is just  
>>>> searching aggressively for old history.
>>>> This may take a while on large repositories
>>>> RA layer request failed: REPORT request failed on '/repos/asf/! 
>>>> svn/vcc/default': REPORT of '/repos/asf/!svn/vcc/default': 403  
>>>> Forbidden (http://svn.apache.org) at /home/oom/tools/libexec/git- 
>>>> core//git-svn line 3955
>> -- 
>> Chris K Wensel
>> chris@wensel.net
>> http://chris.wensel.net/
>> http://www.cascading.org/

--
Chris K Wensel
chris@wensel.net
http://chris.wensel.net/
http://www.cascading.org/

Re: git clone of hadoop core?

Posted by Patrick Hunt <ph...@apache.org>.
Owen O'Malley wrote:
 > Once I have a set of repositories for hadoop core, zookeeper, and pig,
 > I'll try to figure out someplace to post them.


github would be great but it looks like "free for open source" is only 
available up to 100mb. perhaps they would make an exception...

http://github.com/plans

Patrick

Re: git clone of hadoop core?

Posted by Doug Cutting <cu...@apache.org>.
Performance of svn.eu.apache.org should be better since the load is 
generally lower on that server.  Git can place big demands on svn, and 
is thus discouraged on the primary server and encouraged on the eu mirror.

Doug

Owen O'Malley wrote:
> Ok, using the link that Doug sent out, I've managed to clone Zookeeper. 
> (I started there, because it is smaller.) The secret is to use:
>   1. Use svn.eu.apache.org instead of svn.apache.org. I've just sent 
> email to infr@ to ask if we can get the US svn fixed.
>   2. You need to use https and --user to authenticate.
> 
> Once I have a set of repositories for hadoop core, zookeeper, and pig, 
> I'll try to figure out someplace to post them.
> 
> -- Owen

Re: git clone of hadoop core?

Posted by Owen O'Malley <om...@apache.org>.
Ok, using the link that Doug sent out, I've managed to clone  
Zookeeper. (I started there, because it is smaller.) The secret is to  
use:
   1. Use svn.eu.apache.org instead of svn.apache.org. I've just sent  
email to infr@ to ask if we can get the US svn fixed.
   2. You need to use https and --user to authenticate.

Once I have a set of repositories for hadoop core, zookeeper, and pig,  
I'll try to figure out someplace to post them.

-- Owen

Re: git clone of hadoop core?

Posted by Doug Cutting <cu...@apache.org>.
FYI, Jukka Zitting has been exploring using Git with Apache's SVN over 
on the infrastructure-dev@a.o mailing list.

http://markmail.org/message/fzzy7nepk7olx5fl

I don't know how his methods compare to Owen's & Pat's.

Doug

Chris K Wensel wrote:
> Just curious if there has been any progress/changes on this.
> 
> I'm thinking a git repo of Hadoop would be a handy thing. but i expect 
> having every random developer attempt to clone the apache hadoop repo 
> via git-svn would be crushing.
> 
> any thought on how a mirrored git repo would work?
> 
> cheers,
> chris
> 
> On Sep 9, 2008, at 7:04 AM, Patrick Hunt wrote:
> 
>> Owen, you could be running into one of the following issues:
>>
>> Proxy btw you and the server doesn't support the http REPORT command:
>> http://vsingleton.blogspot.com/2008/04/svn-propfind-request-failed-on.html 
>>
>>
>> or perhaps the svn http access is configured to not allow the http 
>> REPORT command:
>> http://svn.haxx.se/users/archive-2005-01/0971.shtml
>>
>> Hope this helps.
>>
>> Patrick
>>
>> Owen O'Malley wrote:
>>> Has anyone succeeded in using git-svn to clone the Hadoop core 
>>> subversion repository with branches? I tried it and it failed. I got 
>>> the following logs. Any suggestions?
>>> -- Owen
>>> % git svn init -s http://svn.apache.org/repos/asf/hadoop/core
>>> Initialized empty Git repository in /export/hadoop-clone/.git/
>>> Using higher level of URL: 
>>> http://svn.apache.org/repos/asf/hadoop/core => 
>>> http://svn.apache.org/repos/asf
>>> % git svn fetch
>>> W: Ignoring error from SVN, path probably does not exist: (160013): 
>>> Filesystem has no item: REPORT request failed on 
>>> '/repos/asf/!svn/bc/100':
>>> File not found: revision 100, path '/hadoop/core'
>>> W: Do not be alarmed at the above message git-svn is just searching 
>>> aggressively for old history.
>>> This may take a while on large repositories
>>> Found possible branch point: 
>>> http://svn.apache.org/repos/asf/lucene/hadoop/trunk => 
>>> http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
>>> Initializing parent: trunk@614596
>>> W: Ignoring error from SVN, path probably does not exist: (160013): 
>>> Filesystem has no item: REPORT request failed on 
>>> '/repos/asf/!svn/bc/101':
>>> File not found: revision 101, path '/lucene/hadoop/trunk'
>>> W: Do not be alarmed at the above message git-svn is just searching 
>>> aggressively for old history.
>>> This may take a while on large repositories
>>> RA layer request failed: REPORT request failed on 
>>> '/repos/asf/!svn/vcc/default': REPORT of 
>>> '/repos/asf/!svn/vcc/default': 403 Forbidden (http://svn.apache.org) 
>>> at /home/oom/tools/libexec/git-core//git-svn line 3955
> 
> -- 
> Chris K Wensel
> chris@wensel.net
> http://chris.wensel.net/
> http://www.cascading.org/
>

Re: git clone of hadoop core?

Posted by Chris K Wensel <ch...@wensel.net>.
Just curious if there has been any progress/changes on this.

I'm thinking a git repo of Hadoop would be a handy thing. but i expect  
having every random developer attempt to clone the apache hadoop repo  
via git-svn would be crushing.

any thought on how a mirrored git repo would work?

cheers,
chris

On Sep 9, 2008, at 7:04 AM, Patrick Hunt wrote:

> Owen, you could be running into one of the following issues:
>
> Proxy btw you and the server doesn't support the http REPORT command:
> http://vsingleton.blogspot.com/2008/04/svn-propfind-request-failed-on.html
>
> or perhaps the svn http access is configured to not allow the http  
> REPORT command:
> http://svn.haxx.se/users/archive-2005-01/0971.shtml
>
> Hope this helps.
>
> Patrick
>
> Owen O'Malley wrote:
>> Has anyone succeeded in using git-svn to clone the Hadoop core  
>> subversion repository with branches? I tried it and it failed. I  
>> got the following logs. Any suggestions?
>> -- Owen
>> % git svn init -s http://svn.apache.org/repos/asf/hadoop/core
>> Initialized empty Git repository in /export/hadoop-clone/.git/
>> Using higher level of URL: http://svn.apache.org/repos/asf/hadoop/core 
>>  => http://svn.apache.org/repos/asf
>> % git svn fetch
>> W: Ignoring error from SVN, path probably does not exist: (160013):  
>> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/ 
>> bc/100':
>> File not found: revision 100, path '/hadoop/core'
>> W: Do not be alarmed at the above message git-svn is just searching  
>> aggressively for old history.
>> This may take a while on large repositories
>> Found possible branch point: http://svn.apache.org/repos/asf/lucene/hadoop/trunk 
>>  => http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
>> Initializing parent: trunk@614596
>> W: Ignoring error from SVN, path probably does not exist: (160013):  
>> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/ 
>> bc/101':
>> File not found: revision 101, path '/lucene/hadoop/trunk'
>> W: Do not be alarmed at the above message git-svn is just searching  
>> aggressively for old history.
>> This may take a while on large repositories
>> RA layer request failed: REPORT request failed on '/repos/asf/!svn/ 
>> vcc/default': REPORT of '/repos/asf/!svn/vcc/default': 403  
>> Forbidden (http://svn.apache.org) at /home/oom/tools/libexec/git- 
>> core//git-svn line 3955

--
Chris K Wensel
chris@wensel.net
http://chris.wensel.net/
http://www.cascading.org/

Re: git clone of hadoop core?

Posted by Patrick Hunt <ph...@apache.org>.
Owen, you could be running into one of the following issues:

Proxy btw you and the server doesn't support the http REPORT command:
http://vsingleton.blogspot.com/2008/04/svn-propfind-request-failed-on.html

or perhaps the svn http access is configured to not allow the http 
REPORT command:
http://svn.haxx.se/users/archive-2005-01/0971.shtml

Hope this helps.

Patrick

Owen O'Malley wrote:
> Has anyone succeeded in using git-svn to clone the Hadoop core 
> subversion repository with branches? I tried it and it failed. I got the 
> following logs. Any suggestions?
> 
> -- Owen
> 
> % git svn init -s http://svn.apache.org/repos/asf/hadoop/core
> Initialized empty Git repository in /export/hadoop-clone/.git/
> Using higher level of URL: http://svn.apache.org/repos/asf/hadoop/core 
> => http://svn.apache.org/repos/asf
> 
> % git svn fetch
> W: Ignoring error from SVN, path probably does not exist: (160013): 
> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/100':
> File not found: revision 100, path '/hadoop/core'
> 
> W: Do not be alarmed at the above message git-svn is just searching 
> aggressively for old history.
> This may take a while on large repositories
> Found possible branch point: 
> http://svn.apache.org/repos/asf/lucene/hadoop/trunk => 
> http://svn.apache.org/repos/asf/hadoop/core/trunk, 614596
> Initializing parent: trunk@614596
> W: Ignoring error from SVN, path probably does not exist: (160013): 
> Filesystem has no item: REPORT request failed on '/repos/asf/!svn/bc/101':
> File not found: revision 101, path '/lucene/hadoop/trunk'
> 
> W: Do not be alarmed at the above message git-svn is just searching 
> aggressively for old history.
> This may take a while on large repositories
> RA layer request failed: REPORT request failed on 
> '/repos/asf/!svn/vcc/default': REPORT of '/repos/asf/!svn/vcc/default': 
> 403 Forbidden (http://svn.apache.org) at 
> /home/oom/tools/libexec/git-core//git-svn line 3955
>