You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@subversion.apache.org by Russell Yanofsky <re...@columbia.edu> on 2003/06/14 02:35:04 UTC

[PATCH] cvs2svn on windows

Changes to cvs2svn.py and its regression testing script so they can work on
Windows. (All the tests pass, by the way, except for #2 "detection of the
executable flag" which must be a real doozie :)

* tools/cvs2svn/cvs2svn.py (relative_name): Changed this helper function to
  expect os.sep in the file path passed to it instead of forward slashes.
  Also added an assertion to detect incorrect usage.

  (add_or_change_path): got rid of the buffer size parameter to the os.popen()
  call which causes a strange error on windows: "ValueError: popen() arg 3
  must be -1"

  (pass3): Changed to sort the file listing in memory instead with the "sort"
  command on Windows. Original behavior is preserved on other platforms.

* tools/cvs2svn/run-tests.py (repos_to_url): Added missing slash detection.
  Needed because absolute paths on windows begin with a drive letter instead
  of a slash.

  (svn_strptime): new function to take the place of time.strptime, which
  doesn't exist on windows

  (Log.__init__): replaced time.strptime() call with svn_strptime() call.


begin 666 cvs2svn.diff
M26YD97@Z('1O;VQS+V-V<S)S=FXO8W9S,G-V;BYP>0H]/3T]/3T]/3T]/3T]
M/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]
M/3T]/3T]/3T]"BTM+2!T;V]L<R]C=G,R<W9N+V-V<S)S=FXN<'D)*')E=FES
M:6]N(#8R,C$I"BLK*R!T;V]L<R]C=G,R<W9N+V-V<S)S=FXN<'D)*'=O<FMI
M;F<@8V]P>2D*0$ @+3(X-"PQ,2 K,C@T+#@@0$ *( H@9&5F(')E;&%T:79E
M7VYA;64H8W9S<F]O="P@9FYA;64I.@H@("!L(#T@;&5N*&-V<W)O;W0I"BT@
M(&EF(&9N86UE6SIL72 ]/2!C=G-R;V]T.@HM(" @(&EF(&9N86UE6VQ=(#T]
M("<O)SH*+2 @(" @(')E='5R;B!F;F%M95ML*S$Z70HM(" @(')E='5R;B!F
M;F%M95ML.ET*+2 @<F5T=7)N(&P**R @87-S97)T(&9N86UE6SIL72 ]/2!C
M=G-R;V]T(&%N9"!F;F%M95ML72 ]/2!O<RYS97 **R @<F5T=7)N('-T<FEN
M9RYR97!L86-E*&9N86UE6VPK,3I=+"!O<RYS97 L("<O)RD*( H@9&5F('9I
M<VET7V9I;&4H87)G+"!D:7)N86UE+"!F:6QE<RDZ"B @(&-D+"!P+"!S=&%T
M<R ](&%R9PI 0" M-S8W+#<@*S<V-"PW($! "B @(" @(R,C('5S92!I="!T
M;R!S970@<W9N.FUI;64M='EP92X*( H@(" @(&)A<V5N86UE(#T@;W,N<&%T
M:"YB87-E;F%M92AR8W-?9FEL95LZ+3)=*0HM(" @('!I<&4@/2!O<RYP;W!E
M;B@G8V\@+7$@+7 E<R!<)R5S7"<G("4@*&-V<U]R978L(')C<U]F:6QE*2P@
M)W(G+" Q,#(T,# I"BL@(" @<&EP92 ](&]S+G!O<&5N*"=C;R M<2 M<"5S
M("(E<R(G("4@*&-V<U]R978L(')C<U]F:6QE*2P@)W(G*0H@"B @(" @(R!9
M;W4@;6EG:'0@=&AI;FL@=V4@8V]U;&0@:G5S="!T97-T"B @(" @(PI 0" M
M,3(W."PW("LQ,C<U+#<@0$ *(" @(R!R96%D('1H92!R97-Y;F,@9&%T82!F
M:6QE"B @(')E<WEN8R ](')E861?<F5S>6YC*&-T>"YL;V=?9FYA;65?8F%S
M92 K(%)%4UE.0U]3549&25@I"B *+2 @;W5T<'5T(#T@;W!E;BAC='@N;&]G
M7V9N86UE7V)A<V4@*R!#3$5!3E]215937U-51D9)6"P@)W<G*0HK("!O=71P
M=70@/2!O<&5N*&-T>"YL;V=?9FYA;65?8F%S92 K($-,14%.7U)%5E-?4U5&
M1DE8+" G=W0G*0H@"B @(",@<')O8V5S<R!T:&4@<F5V:7-I;VYS(&9I;&4L
M(&QO;VMI;F<@9F]R(&ET96US('1O(&-L96%N('5P"B @(&9O<B!L:6YE(&EN
M(&9I;&5I;G!U="Y&:6QE26YP=70H8W1X+FQO9U]F;F%M95]B87-E("L@4D56
M4U]3549&25@I.@I 0" M,3,Q,RPX("LQ,S$P+#$T($! "B *(&1E9B!P87-S
M,RAC='@I.@H@(" C('-O<G0@=&AE(&QO9R!F:6QE<PHM("!O<RYS>7-T96TH
M)W-O<G0@)7,@/B E<R<@)2 H8W1X+FQO9U]F;F%M95]B87-E("L@0TQ%04Y?
M4D564U]3549&25@L"BT@(" @(" @(" @(" @(" @(" @(" @(" @(" @("!C
M='@N;&]G7V9N86UE7V)A<V4@*R!33U)4141?4D564U]3549&25@I*0HK("!C
M;&5A;B ](&-T>"YL;V=?9FYA;65?8F%S92 K($-,14%.7U)%5E-?4U5&1DE8
M"BL@('-O<G1E9" ](&-T>"YL;V=?9FYA;65?8F%S92 K(%-/4E1%1%]21593
M7U-51D9)6 HK("!I9B!S>7,N<&QA=&9O<FT@/3T@(G=I;C,R(CH**R @("!L
M:6YE<R ](&]P96XH8VQE86XL("=R="<I+G)E861L:6YE<R@I"BL@(" @;&EN
M97,N<V]R="@I"BL@(" @;W!E;BAS;W)T960L("=W="<I+G=R:71E;&EN97,H
M;&EN97,I"BL@(&5L<V4Z"BL@(" @;W,N<WES=&5M*"=S;W)T("5S(#X@)7,G
M("4@*&-L96%N+"!S;W)T960I*0H@"B!D968@<&%S<S0H8W1X*3H*(" @(R!C
M<F5A=&4@=&AE('1A<F=E="!R97!O<VET;W)Y"DEN9&5X.B!T;V]L<R]C=G,R
M<W9N+W)U;BUT97-T<RYP>0H]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]
M/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]/3T]"BTM+2!T
M;V]L<R]C=G,R<W9N+W)U;BUT97-T<RYP>0DH<F5V:7-I;VX@-C(R,2D**RLK
M('1O;VQS+V-V<S)S=FXO<G5N+71E<W1S+G!Y"2AW;W)K:6YG(&-O<'DI"D! 
M("TX-2PX("LX-2PR,"! 0 H@"B!D968@<F5P;W-?=&]?=7)L*'!A=&A?=&]?
M<W9N7W)E<&]S*3H*(" @(B(B5&AI<R!D;V5S('=H870@>6]U('1H:6YK(&ET
M(&1O97,N(B(B"BT@(')E='5R;B G9FEL93HO+R5S)R E(&]S+G!A=&@N86)S
M<&%T:"AP871H7W1O7W-V;E]R97!O<RD**R @<G!A=&@@/2!O<RYP871H+F%B
M<W!A=&@H<&%T:%]T;U]S=FY?<F5P;W,I"BL@(&EF(')P871H6S!=("$]("<O
M)SH**R @("!R<&%T:" ]("<O)R K(')P871H"BL@(')E='5R;B G9FEL93HO
M+R5S)R E(')P871H"B **VEF(&AA<V%T='(H=&EM92P@)W-T<G!T:6UE)RDZ
M"BL@(&1E9B!S=FY?<W1R<'1I;64H=&EM97-T<BDZ"BL@(" @<F5T=7)N('1I
M;64N<W1R<'1I;64H=&EM97-T<BP@)R59+25M+25D("5(.B5-.B53)RD**V5L
M<V4Z"BL@(%]R95]R979?9&%T92 ](')E+F-O;7!I;&4H)RA;,"TY77LT?2DM
M*%LP+3E=6S M.5TI+2A;,"TY75LP+3E=*2 G"BL@(" @(" @(" @(" @(" @
M(" @(" @(" @(" @)RA;,"TY75LP+3E=*3HH6S M.5U;,"TY72DZ*%LP+3E=
M6S M.5TI)RD**R @9&5F('-V;E]S=')P=&EM92AT:6UE<W1R*3H**R @("!M
M871C:&5S(#T@7W)E7W)E=E]D871E+FUA=&-H*'1I;65S='(I+F=R;W5P<R@I
M"BL@(" @<F5T=7)N('1U<&QE*&UA<"AI;G0L(&UA=&-H97,I*2 K("@P+" Q
M+" P*0H@"B!C;&%S<R!,;V<Z"B @(&1E9B!?7VEN:71?7RAS96QF+"!R979I
M<VEO;BP@875T:&]R+"!D871E*3H*0$ @+3$P,"PW("LQ,3(L-R! 0 H@(" @
M(",*(" @(" C(&%N9"!T:6UE+FUK=&EM92@I(&-O;G9E<G1S(&9R;VT@;&]C
M86QT:6UE+"!I="!A;&P@=V]R:W,@;W5T('9E<GD*(" @(" C(&AA<'!I;'DN
M"BT@(" @<V5L9BYD871E(#T@=&EM92YM:W1I;64H=&EM92YS=')P=&EM92AD
M871E6S Z,3E=+" B)5DM)6TM)60@)4@Z)4TZ)5,B*2D**R @("!S96QF+F1A
M=&4@/2!T:6UE+FUK=&EM92AS=FY?<W1R<'1I;64H9&%T95LP.C$Y72DI"B *
M(" @(" C(%1H92!C:&%N9V5D('!A=&AS('=I;&P@8F4@86-C=6UU;&%T960@
M;&%T97(L(&%S(&QO9R!D871A(&ES(')E860N"B @(" @(R!+97ES(&AE<F4@
M87)E('!A=&AS('-U8V@@87,@)R]T<G5N:R]F;V\O8F%R)RP@=F%L=65S(&%R
)92!L971T97(*
`
end



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [PATCH] cvs2svn on windows

Posted by kf...@collab.net.

"Russell Yanofsky" <re...@columbia.edu> writes:
> Changes to cvs2svn.py and its regression testing script so they can work on
> Windows. (All the tests pass, by the way, except for #2 "detection of the
> executable flag" which must be a real doozie :)

Thank you!

Sander R, I'll file this one myself.

-K


> * tools/cvs2svn/cvs2svn.py (relative_name): Changed this helper function to
>   expect os.sep in the file path passed to it instead of forward slashes.
>   Also added an assertion to detect incorrect usage.
> 
>   (add_or_change_path): got rid of the buffer size parameter to the os.popen()
>   call which causes a strange error on windows: "ValueError: popen() arg 3
>   must be -1"
> 
>   (pass3): Changed to sort the file listing in memory instead with the "sort"
>   command on Windows. Original behavior is preserved on other platforms.
> 
> * tools/cvs2svn/run-tests.py (repos_to_url): Added missing slash detection.
>   Needed because absolute paths on windows begin with a drive letter instead
>   of a slash.
> 
>   (svn_strptime): new function to take the place of time.strptime, which
>   doesn't exist on windows
> 
>   (Log.__init__): replaced time.strptime() call with svn_strptime() call.
> 
> 
> Index: tools/cvs2svn/cvs2svn.py
> ===================================================================
> --- tools/cvs2svn/cvs2svn.py	(revision 6221)
> +++ tools/cvs2svn/cvs2svn.py	(working copy)
> @@ -284,11 +284,8 @@
>  
>  def relative_name(cvsroot, fname):
>    l = len(cvsroot)
> -  if fname[:l] == cvsroot:
> -    if fname[l] == '/':
> -      return fname[l+1:]
> -    return fname[l:]
> -  return l
> +  assert fname[:l] == cvsroot and fname[l] == os.sep
> +  return string.replace(fname[l+1:], os.sep, '/')
>  
>  def visit_file(arg, dirname, files):
>    cd, p, stats = arg
> @@ -767,7 +764,7 @@
>      ### use it to set svn:mime-type.
>  
>      basename = os.path.basename(rcs_file[:-2])
> -    pipe = os.popen('co -q -p%s \'%s\'' % (cvs_rev, rcs_file), 'r', 102400)
> +    pipe = os.popen('co -q -p%s "%s"' % (cvs_rev, rcs_file), 'r')
>  
>      # You might think we could just test
>      #
> @@ -1278,7 +1275,7 @@
>    # read the resync data file
>    resync = read_resync(ctx.log_fname_base + RESYNC_SUFFIX)
>  
> -  output = open(ctx.log_fname_base + CLEAN_REVS_SUFFIX, 'w')
> +  output = open(ctx.log_fname_base + CLEAN_REVS_SUFFIX, 'wt')
>  
>    # process the revisions file, looking for items to clean up
>    for line in fileinput.FileInput(ctx.log_fname_base + REVS_SUFFIX):
> @@ -1313,8 +1310,14 @@
>  
>  def pass3(ctx):
>    # sort the log files
> -  os.system('sort %s > %s' % (ctx.log_fname_base + CLEAN_REVS_SUFFIX,
> -                              ctx.log_fname_base + SORTED_REVS_SUFFIX))
> +  clean = ctx.log_fname_base + CLEAN_REVS_SUFFIX
> +  sorted = ctx.log_fname_base + SORTED_REVS_SUFFIX
> +  if sys.platform == "win32":
> +    lines = open(clean, 'rt').readlines()
> +    lines.sort()
> +    open(sorted, 'wt').writelines(lines)
> +  else:
> +    os.system('sort %s > %s' % (clean, sorted))
>  
>  def pass4(ctx):
>    # create the target repository
> Index: tools/cvs2svn/run-tests.py
> ===================================================================
> --- tools/cvs2svn/run-tests.py	(revision 6221)
> +++ tools/cvs2svn/run-tests.py	(working copy)
> @@ -85,8 +85,20 @@
>  
>  def repos_to_url(path_to_svn_repos):
>    """This does what you think it does."""
> -  return 'file://%s' % os.path.abspath(path_to_svn_repos)
> +  rpath = os.path.abspath(path_to_svn_repos)
> +  if rpath[0] != '/':
> +    rpath = '/' + rpath
> +  return 'file://%s' % rpath
>  
> +if hasattr(time, 'strptime'):
> +  def svn_strptime(timestr):
> +    return time.strptime(timestr, '%Y-%m-%d %H:%M:%S')
> +else:
> +  _re_rev_date = re.compile('([0-9]{4})-([0-9][0-9])-([0-9][0-9]) '
> +                            '([0-9][0-9]):([0-9][0-9]):([0-9][0-9])')
> +  def svn_strptime(timestr):
> +    matches = _re_rev_date.match(timestr).groups()
> +    return tuple(map(int, matches)) + (0, 1, 0)
>  
>  class Log:
>    def __init__(self, revision, author, date):
> @@ -100,7 +112,7 @@
>      #
>      # and time.mktime() converts from localtime, it all works out very
>      # happily.
> -    self.date = time.mktime(time.strptime(date[0:19], "%Y-%m-%d %H:%M:%S"))
> +    self.date = time.mktime(svn_strptime(date[0:19]))
>  
>      # The changed paths will be accumulated later, as log data is read.
>      # Keys here are paths such as '/trunk/foo/bar', values are letter
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [PATCH] cvs2svn on windows (updated resend)

Posted by Russell Yanofsky <re...@columbia.edu>.

Russell Yanofsky wrote:
> cmpilato@collab.net wrote:
>> "Russell Yanofsky" <re...@columbia.edu> writes:
>>>   (pass3): Changed to sort the file listing in memory instead of
>>>   with the "sort" command on Windows. Original behavior is
>>>   preserved on other platforms.
>>
>> I'm confused.  Windows has a 'sort' program that supports path input
>> and redirected I/O.  Why not use it?  Does it behave differently than
>> it should?
>
> The windows sort program does seem to work some times, but it fails
> at other times.
> ...
> I don't remember what causes this, but I'll look into it.

Turns out the problem running the sort command was not in cvs2svn.py, but in
run-tests.py.

When you ran cvs2svn.py directly from the command line it would work, but if it
were run under run-tests.py, it wouldn't work. I experimented with the code in
run-tests.py that starts up cvs2svn, and found that changing the comand line
from

  cvs2svn.py [args]

to

  python cvs2svn.py [args]

made everything work. A new patch is attached that has this change and the
original sorting code.

<aside>
I couldn't figure out what the difference between the two invocations is, but I
did write a little script to reproduce the problem:

  import os, sys

  cmdline = "python %s arg" % sys.argv[0]  # working command line
  #cmdline = "%s arg" % sys.argv[0]        # broken command line

  if len(sys.argv) == 1:
    infile, outfile, errfile = os.popen3(cmdline, 't')
    print "".join(outfile.readlines())
  else:
    if os.system("echo WORKS"):
      print "BROKEN"
</aside>

Log Message:

Changes to cvs2svn.py and its regression testing script so they will
work on Windows.

* tools/cvs2svn/cvs2svn.py (relative_name): Changed this function to
  expect os.sep in the file path passed to it instead of forward
  slashes. Also added an assertion to detect incorrect usage.

  (escape_shell_arg): new function to escape shell arguments, works
  differently on unix and windows

  (Dumper.add_or_change_path): changed to check out files in binary
  mode, and to use the escape_shell_arg function

* tools/cvs2svn/run-tests.py (repos_to_url): Added missing slash
  detection. Needed because absolute paths on windows begin with
  a drive letter instead of a slash.

  (svn_strptime): new function to take the place of time.strptime,
  which doesn't exist on windows

  (Log.__init__): replaced time.strptime() call with svn_strptime()
  call.

  (run_cvs2svn): changed command line used to start cvs2svn on windows

Re: [PATCH] cvs2svn on windows (resend)

Posted by Russell Yanofsky <re...@columbia.edu>.

cmpilato@collab.net wrote:
> "Russell Yanofsky" <re...@columbia.edu> writes:
>
>> Log Message:
>>
>> Changes to cvs2svn.py and its regression testing script so they will
>> work on Windows.
>
> [...]
>
>>   (pass3): Changed to sort the file listing in memory instead of with
>>   the "sort" command on Windows. Original behavior is preserved on
>>   other platforms.
>
> I'm confused.  Windows has a 'sort' program that supports path input
> and redirected I/O.  Why not use it?  Does it behave differently than
> it should?

The windows sort program does seem to work some times, but it fails at other
times. Here are the test results when the sort program is used:

M:\russ\source\svn\tools\cvs2svn>python run-tests.py
PASS:  run-tests.py 1: cvs2svn with no arguments shows usage
PASS:  run-tests.py 2: fail early on encountering an invalid symbolic name
PASS:  run-tests.py 3: fail early on encountering a branch with two names
FAIL:  run-tests.py 4: convert as much as can, despite a corrupt ,v file
FAIL:  run-tests.py 5: convert a branch file rooted in a 'dead' revision
FAIL:  run-tests.py 6: detection of the executable flag
FAIL:  run-tests.py 7: conversion of filename with a space
FAIL:  run-tests.py 8: two commits in quick succession
FAIL:  run-tests.py 9: prune, but never too much
FAIL:  run-tests.py 10: file deleted twice, in the root of the repository
FAIL:  run-tests.py 11: simple trunk commits
FAIL:  run-tests.py 12: two interleaved trunk commits, with different log msgs
XFAIL: run-tests.py 13: simple tags
XFAIL: run-tests.py 14: simple branch commits
XFAIL: run-tests.py 15: a commit affecting both trunk and a branch
FAIL:  run-tests.py 16: branch created from both trunk and from another branch
FAIL:  run-tests.py 17: resyncing should not put commit groups in the wrong
order

And here are the results when in memory sorting is used:

M:\russ\source\svn\tools\cvs2svn>python run-tests.py
PASS:  run-tests.py 1: cvs2svn with no arguments shows usage
PASS:  run-tests.py 2: fail early on encountering an invalid symbolic name
PASS:  run-tests.py 3: fail early on encountering a branch with two names
PASS:  run-tests.py 4: convert as much as can, despite a corrupt ,v file
PASS:  run-tests.py 5: convert a branch file rooted in a 'dead' revision
FAIL:  run-tests.py 6: detection of the executable flag
PASS:  run-tests.py 7: conversion of filename with a space
PASS:  run-tests.py 8: two commits in quick succession
PASS:  run-tests.py 9: prune, but never too much
PASS:  run-tests.py 10: file deleted twice, in the root of the repository
PASS:  run-tests.py 11: simple trunk commits
PASS:  run-tests.py 12: two interleaved trunk commits, with different log msgs
XFAIL: run-tests.py 13: simple tags
XFAIL: run-tests.py 14: simple branch commits
XFAIL: run-tests.py 15: a commit affecting both trunk and a branch
PASS:  run-tests.py 16: branch created from both trunk and from another branch
PASS:  run-tests.py 17: resyncing should not put commit groups in the wrong
order

I don't remember what causes this, but I'll look into it.

- Russ



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org