You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Lawrence Stewart <ls...@room52.net> on 2008/08/24 05:30:39 UTC
Help with Python SWIG bindings and callbacks
Hi all,
I'm playing with the Python SWIG bindings for SVN 1.5.1 and have run
into a brick wall. I want to update a working copy from a python script,
and have a python function callback executed for each item updated. The
python code below hopefully illustrates what I'm trying to do.
def update_callback(notify, pool):
print "in callback"
def update(wc, rev):
ctx = svn.client.svn_client_create_context()
ctx.notify_func2 = update_callback
r = svn.core.svn_opt_revision_t()
r.kind = svn.core.svn_opt_revision_head
result_rev = svn.client.svn_client_update(wc, r, True, ctx)
print "result_rev: %d" % result_rev
The update function does work and my working copy does get updated the
HEAD, but my callback is never executed.
I'm basing this code on the statement in
http://svn.collab.net/svn-doxygen/group__Update.html that says if
ctx->notify_func2 is not NULL it will be called for each item updated.
This of course may not work or be applicable in Python land... someone
please tell me if this is the case.
I've tried countless variations of the above code and changing
parameters to the callback, changing the way the callback is assigned to
the context object (I tried following the way it was done in the
client.py unit test file for a different callback type that is used)
etc. etc. without success.
I suspect my problem is that I just don't grok the translation that SWIG
is doing from the C API up into Python and therein lies the problem.
Any help in understanding what I'm doing wrong or how I can achieve
something equivalent would be greatly appreciated.
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by David James <ja...@gmail.com>.
On Mon, Aug 25, 2008 at 7:33 PM, Lawrence Stewart <ls...@room52.net> wrote:
>>> One really nice part about the ctypes python bindings is that you
>>> don't need to recompile the bindings for it to work with a different
>>> version of Subversion, or a different platform -- it's a pure python
>>> binding.
>
> Sounds too good to be true! If I'm understanding you correctly, the ctypes
> python module does the necessary magic to tie the low level python-svn API
> code to the underlying raw SVN C APIs without any SVN C code glue? Then you
> just layer some additional Python classes on top and that's it?
Yup. In fact, you don't even need to have any Python classes on top --
you can just use the raw C APIs if you want, directly from Python. The
ctypes-python-bindings use ctypesgen
(http://code.google.com/p/ctypesgen/) to generate ctypes definitions,
which tell ctypes what functions are in the files, and what the types
of each argument are, so that you can access the C functions and C
structs easily from within python without needing to do any
declarations yourself.
The high level Python classes in the ctypes-python-bindings are there
for convenience only, for the most common tasks you need to do with
the SVN API. If you have tasks that the convenience API doesn't
support it might be a good idea to write some more classes and
contribute them back to the ctypes python bindings. But strictly
speaking that isn't necessary, because you can just use the low level
bindings.
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by Lawrence Stewart <ls...@room52.net>.
David James wrote:
> On Mon, Aug 25, 2008 at 7:02 PM, David James <ja...@gmail.com> wrote:
>> On Mon, Aug 25, 2008 at 6:18 PM, Lawrence Stewart <ls...@room52.net> wrote:
>>>> Have you considered trying out the new ctypes python bindings? I tried
>>>> converting your script to use the new ctypes python bindings, and it
>>>> seems to work fine. See the example below.
>>>>
>>>> from csvn.core import *
>>>> import csvn.wc
>>>> svn_cmdline_init("", csvn.core.stderr)
>>>> wc = csvn.wc.WC("wc")
>>>>
>>>> def notify_func(obj):
>>>> if obj.action == svn_wc_notify_update_update:
>>>> print "U", obj.path
>>>> elif obj.action == svn_wc_notify_update_completed:
>>>> print "Completed", obj.path
>>>>
>>>> wc.checkout("http://svn.collab.net/repos/svn/trunk/www/")
>>>> wc.update(revnum=32695)
>>>> wc.set_notify_func(notify_func)
>>>> wc.update(revnum=32696)
>>>>
>>>> The above example works on my machine, and outputs the following:
>>>> U wc/faq.ja.html
>>>> U wc
>>>> Completed wc
>>>>
>>>> The ctypes python bindings are available at
>>>> http://svn.collab.net/repos/svn/branches/ctypes-python-bindings/, and
>>>> should work fine with Subversion 1.5.x. Hopefully they will be
>>>> standard in Subversion 1.6.0, but we are still working on a few
>>>> finishing touches.
>>>
>>> No I haven't considered ctypes (wasn't even aware of it). Thanks for the
>>> pointer. Is there other documentation than the README in the top level
>>> branch dir that I should be referring to?
>>>
>>> Is it anticipated that the ctypes interface will usurp the SWIG code? I had
>>> kind of hoped to create a script that was backward compatible with vanilla
>>> SVN back to 1.4.x hence the use of the SWIG API and not something like
>>> PySVN, but I guess it's not disastrous if that's not the case.
>>>
>>> As a bit of background, I'm trying to create a script that allows me to
>>> mirror a portion of a remote repo into my own development repo as a vendor
>>> branch. The existing tools just don't seem to cut it for this use case and
>>> hacking the svn C code seemed a bit extreme to get something working
>>> relatively quickly. Given the trouble I'm having with the bindings perhaps I
>>> should have just dived straight into writing a small C app. I'm willing to
>>> be told otherwise if someone here has/knows such a tool in which case my
>>> need for python bindings disappears :)
>> Hi Lawrence,
>>
>> The ctypes python bindings should work with vanilla Subversion 1.4.x
>> or later. I haven't tried them recently with Subversion 1.4, but they
>> should work. If they don't still work with SVN 1.4, let me know.
Awesome!
>>
>> One really nice part about the ctypes python bindings is that you
>> don't need to recompile the bindings for it to work with a different
>> version of Subversion, or a different platform -- it's a pure python
>> binding.
Sounds too good to be true! If I'm understanding you correctly, the
ctypes python module does the necessary magic to tie the low level
python-svn API code to the underlying raw SVN C APIs without any SVN C
code glue? Then you just layer some additional Python classes on top and
that's it?
>>
>> If you like Python, the ctypes python bindings are probably the
>> easiest way to get started with the Subversion API. The low level APIs
>> in ctypes don't have documentation, but they don't need it, because
>> they are an exact mirror of the C APIs. All C APIs work verbatim
>> identical in the ctypes python bindings.
I certainly like the sound of it. I grok the svn C APIs just fine... it
was the SWIG translation code that was doing my head in. This ctypes
stuff sounds great (even if it is Python specific and therefore not
applicable to all languages).
>>
>> We also implemented some higher level python objects which make it
>> easier to do common tasks in the ctypes python bindings. I find these
>> quite helpful. You can learn the high level APIs by looking at the
>> test suite for the ctypes python bindings. If you want examples of how
>> to use the low level APIs, try taking a look at the implementation of
>> the low level APIs.
Yup cool.
>
> Oops. Typo here. I should have said, if you want examples of how to
> use the low level APIs as well, try taking a look at the
> implementation of the high level APIs, which make use of the low level
> APIs.
rgr.
Thanks for the insights!
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by David James <ja...@gmail.com>.
On Mon, Aug 25, 2008 at 7:02 PM, David James <ja...@gmail.com> wrote:
> On Mon, Aug 25, 2008 at 6:18 PM, Lawrence Stewart <ls...@room52.net> wrote:
>>> Have you considered trying out the new ctypes python bindings? I tried
>>> converting your script to use the new ctypes python bindings, and it
>>> seems to work fine. See the example below.
>>>
>>> from csvn.core import *
>>> import csvn.wc
>>> svn_cmdline_init("", csvn.core.stderr)
>>> wc = csvn.wc.WC("wc")
>>>
>>> def notify_func(obj):
>>> if obj.action == svn_wc_notify_update_update:
>>> print "U", obj.path
>>> elif obj.action == svn_wc_notify_update_completed:
>>> print "Completed", obj.path
>>>
>>> wc.checkout("http://svn.collab.net/repos/svn/trunk/www/")
>>> wc.update(revnum=32695)
>>> wc.set_notify_func(notify_func)
>>> wc.update(revnum=32696)
>>>
>>> The above example works on my machine, and outputs the following:
>>> U wc/faq.ja.html
>>> U wc
>>> Completed wc
>>>
>>> The ctypes python bindings are available at
>>> http://svn.collab.net/repos/svn/branches/ctypes-python-bindings/, and
>>> should work fine with Subversion 1.5.x. Hopefully they will be
>>> standard in Subversion 1.6.0, but we are still working on a few
>>> finishing touches.
>>
>>
>> No I haven't considered ctypes (wasn't even aware of it). Thanks for the
>> pointer. Is there other documentation than the README in the top level
>> branch dir that I should be referring to?
>>
>> Is it anticipated that the ctypes interface will usurp the SWIG code? I had
>> kind of hoped to create a script that was backward compatible with vanilla
>> SVN back to 1.4.x hence the use of the SWIG API and not something like
>> PySVN, but I guess it's not disastrous if that's not the case.
>>
>> As a bit of background, I'm trying to create a script that allows me to
>> mirror a portion of a remote repo into my own development repo as a vendor
>> branch. The existing tools just don't seem to cut it for this use case and
>> hacking the svn C code seemed a bit extreme to get something working
>> relatively quickly. Given the trouble I'm having with the bindings perhaps I
>> should have just dived straight into writing a small C app. I'm willing to
>> be told otherwise if someone here has/knows such a tool in which case my
>> need for python bindings disappears :)
>
> Hi Lawrence,
>
> The ctypes python bindings should work with vanilla Subversion 1.4.x
> or later. I haven't tried them recently with Subversion 1.4, but they
> should work. If they don't still work with SVN 1.4, let me know.
>
> One really nice part about the ctypes python bindings is that you
> don't need to recompile the bindings for it to work with a different
> version of Subversion, or a different platform -- it's a pure python
> binding.
>
> If you like Python, the ctypes python bindings are probably the
> easiest way to get started with the Subversion API. The low level APIs
> in ctypes don't have documentation, but they don't need it, because
> they are an exact mirror of the C APIs. All C APIs work verbatim
> identical in the ctypes python bindings.
>
> We also implemented some higher level python objects which make it
> easier to do common tasks in the ctypes python bindings. I find these
> quite helpful. You can learn the high level APIs by looking at the
> test suite for the ctypes python bindings. If you want examples of how
> to use the low level APIs, try taking a look at the implementation of
> the low level APIs.
Oops. Typo here. I should have said, if you want examples of how to
use the low level APIs as well, try taking a look at the
implementation of the high level APIs, which make use of the low level
APIs.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by David James <ja...@gmail.com>.
On Mon, Aug 25, 2008 at 6:18 PM, Lawrence Stewart <ls...@room52.net> wrote:
>> Have you considered trying out the new ctypes python bindings? I tried
>> converting your script to use the new ctypes python bindings, and it
>> seems to work fine. See the example below.
>>
>> from csvn.core import *
>> import csvn.wc
>> svn_cmdline_init("", csvn.core.stderr)
>> wc = csvn.wc.WC("wc")
>>
>> def notify_func(obj):
>> if obj.action == svn_wc_notify_update_update:
>> print "U", obj.path
>> elif obj.action == svn_wc_notify_update_completed:
>> print "Completed", obj.path
>>
>> wc.checkout("http://svn.collab.net/repos/svn/trunk/www/")
>> wc.update(revnum=32695)
>> wc.set_notify_func(notify_func)
>> wc.update(revnum=32696)
>>
>> The above example works on my machine, and outputs the following:
>> U wc/faq.ja.html
>> U wc
>> Completed wc
>>
>> The ctypes python bindings are available at
>> http://svn.collab.net/repos/svn/branches/ctypes-python-bindings/, and
>> should work fine with Subversion 1.5.x. Hopefully they will be
>> standard in Subversion 1.6.0, but we are still working on a few
>> finishing touches.
>
>
> No I haven't considered ctypes (wasn't even aware of it). Thanks for the
> pointer. Is there other documentation than the README in the top level
> branch dir that I should be referring to?
>
> Is it anticipated that the ctypes interface will usurp the SWIG code? I had
> kind of hoped to create a script that was backward compatible with vanilla
> SVN back to 1.4.x hence the use of the SWIG API and not something like
> PySVN, but I guess it's not disastrous if that's not the case.
>
> As a bit of background, I'm trying to create a script that allows me to
> mirror a portion of a remote repo into my own development repo as a vendor
> branch. The existing tools just don't seem to cut it for this use case and
> hacking the svn C code seemed a bit extreme to get something working
> relatively quickly. Given the trouble I'm having with the bindings perhaps I
> should have just dived straight into writing a small C app. I'm willing to
> be told otherwise if someone here has/knows such a tool in which case my
> need for python bindings disappears :)
Hi Lawrence,
The ctypes python bindings should work with vanilla Subversion 1.4.x
or later. I haven't tried them recently with Subversion 1.4, but they
should work. If they don't still work with SVN 1.4, let me know.
One really nice part about the ctypes python bindings is that you
don't need to recompile the bindings for it to work with a different
version of Subversion, or a different platform -- it's a pure python
binding.
If you like Python, the ctypes python bindings are probably the
easiest way to get started with the Subversion API. The low level APIs
in ctypes don't have documentation, but they don't need it, because
they are an exact mirror of the C APIs. All C APIs work verbatim
identical in the ctypes python bindings.
We also implemented some higher level python objects which make it
easier to do common tasks in the ctypes python bindings. I find these
quite helpful. You can learn the high level APIs by looking at the
test suite for the ctypes python bindings. If you want examples of how
to use the low level APIs, try taking a look at the implementation of
the low level APIs.
It's certainly possible to get the SWIG bindings working and doing
everything you need, but I find them kind of difficult to use because
you need to do so many workarounds to get them working. The ctypes
python bindings are refreshing in comparison because they work
consistently and reliably, just like the underlying C APis.
Hope this helps,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Mirroring subtree of a repo into subtree of another repo (was Re:
Help with Python SWIG bindings and callbacks)
Posted by Lawrence Stewart <ls...@room52.net>.
Hi Blair,
(I'm changing the subject to reflect the different question we're now
discussing)
Blair Zajac wrote:
> Lawrence Stewart wrote:
>> As a bit of background, I'm trying to create a script that allows me
>> to mirror a portion of a remote repo into my own development repo as a
>> vendor branch. The existing tools just don't seem to cut it for this
>> use case and hacking the svn C code seemed a bit extreme to get
>> something working relatively quickly. Given the trouble I'm having
>> with the bindings perhaps I should have just dived straight into
>> writing a small C app. I'm willing to be told otherwise if someone
>> here has/knows such a tool in which case my need for python bindings
>> disappears :)
>
> Have you taken a look at svk? This does most of this already.
>
> http://svk.bestpractical.com/view/HomePage
Yes I've used SVK and it is indeed a great tool. However it requires me
to mirror the entire remote repo which is a pain and it doesn't allow us
to keep our centralised svn infrastructure ticking along.
To paint a picture of the use case I'm trying to work towards... I do
FreeBSD development work and run a svnsync mirror of the offial FreeBSD
svn repo (180000+ commits) at work. Because of the way svnsync works,
this repo is obviously read only. I (along with other colleagues) have
various projects related to FreeBSD we're working on so we have a second
repo which we do our freebsd development work in. It's structured as:
/
|
|-project1/
| |
| |-trunk/
| |
| |-8.x
| |
| |-7.x
|
|-project2/
| |
| |-trunk/
| |
| |-8.x
|
|-vendor/
| |
| |-freebsd/
| |
| |-8.x
| |
| |-7.x
We create projects by branching off the appropriate freebsd X.X vendor
branch, and create our patches to pass upstream to the freebsd guys by
diffing our branch against the vendor branch we branched off.
The freebsd source code is very large (100+MB and thousands of files) so
having a shared vendor branch between all our freebsd projects is a huge
win.
We periodically import snapshots of our freebsd mirror repo into our
local development repo's vendor subtree and merge changes back into our
project branches, but this is a pain and we loose granularity of
information in the FreeBSD commits, the ability to roll back/forward etc
etc. What I want is a script I can run that will do a blow-by-blow
replay of commits into each vendor branch so I can track each commit
that went on in the upstream FreeBSD repository.
I currently use a ruby tool called Piston (http://piston.rubyforge.org/)
which is the best I've found so far. It is however not ideal for working
on code bases the size of FreeBSD. It also only does batched commits
which is a pain (yes, I can make it "batch" on a commit by commit basis
but it's incredibly slow and is fragile because it parses the output of
the svn cmd line tools instead of usng the APIs). A tool that made use
of the SVN APIs directly would do a much better job, hence my attempt to
try and hack up a script to do this.
Alternatively, if there is work in progress to allow an svnsync mirror
to become read/write capable e.g. define a subdir that is the read only
mirror of the upstream repo and all other parts of the local repo can be
written to, I'd be happy to contribute to that effort.
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by Blair Zajac <bl...@orcaware.com>.
Lawrence Stewart wrote:
> As a bit of background, I'm trying to create a script that allows me to
> mirror a portion of a remote repo into my own development repo as a
> vendor branch. The existing tools just don't seem to cut it for this use
> case and hacking the svn C code seemed a bit extreme to get something
> working relatively quickly. Given the trouble I'm having with the
> bindings perhaps I should have just dived straight into writing a small
> C app. I'm willing to be told otherwise if someone here has/knows such a
> tool in which case my need for python bindings disappears :)
Have you taken a look at svk? This does most of this already.
http://svk.bestpractical.com/view/HomePage
Regards,
Blair
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by Lawrence Stewart <ls...@room52.net>.
David James wrote:
> On Sun, Aug 24, 2008 at 6:25 PM, Lawrence Stewart <ls...@room52.net> wrote:
>> Lawrence Stewart wrote:
>>> Lawrence Stewart wrote:
>>>> Hi all,
>>>>
>>>> I'm playing with the Python SWIG bindings for SVN 1.5.1 and have run into
>>>> a brick wall. I want to update a working copy from a python script, and have
>>>> a python function callback executed for each item updated. The python code
>>>> below hopefully illustrates what I'm trying to do.
>>>>
>>>> def update_callback(notify, pool):
>>>> print "in callback"
>>>>
>>>> def update(wc, rev):
>>>> ctx = svn.client.svn_client_create_context()
>>>> ctx.notify_func2 = update_callback
>>>> r = svn.core.svn_opt_revision_t()
>>>> r.kind = svn.core.svn_opt_revision_head
>>>> result_rev = svn.client.svn_client_update(wc, r, True, ctx)
>>>> print "result_rev: %d" % result_rev
>>>>
>>>> The update function does work and my working copy does get updated the
>>>> HEAD, but my callback is never executed.
>>>>
>>>> I'm basing this code on the statement in
>>>> http://svn.collab.net/svn-doxygen/group__Update.html that says if
>>>> ctx->notify_func2 is not NULL it will be called for each item updated. This
>>>> of course may not work or be applicable in Python land... someone please
>>>> tell me if this is the case.
>>>>
>>>> I've tried countless variations of the above code and changing parameters
>>>> to the callback, changing the way the callback is assigned to the context
>>>> object (I tried following the way it was done in the client.py unit test
>>>> file for a different callback type that is used) etc. etc. without success.
>>>>
>>>> I suspect my problem is that I just don't grok the translation that SWIG
>>>> is doing from the C API up into Python and therein lies the problem.
>>>>
>>>> Any help in understanding what I'm doing wrong or how I can achieve
>>>> something equivalent would be greatly appreciated.
>>>
>>> Quick follow up... I just stumbled across this thread that describes what
>>> I'm trying to do.
>>>
>>> http://osdir.com/ml/programming.swig/2003-06/msg00058.html
>>>
>>> It provided the missing clue I needed to make this work.
>>>
>>> Apologies for the noise.
>>
>> Ok, so I can now get the callback to fire, but the arguments passed to it
>> look like random pointer junk and python coredumps somewhere within the
>> svn.client.svn_client_update3() call after having executed the update and a
>> few callbacks. Here's the new code:
>>
>>
>>
>> def update_callback(path, action, kind, mime_type, content_state,
>> prop_state, revision):
>>
>> print "callback\taction: %d\trev: %u" % (action,revision)
>>
>>
>>
>> def update(wc, rev):
>>
>> ctx = svn.client.svn_client_ctx_t()
>>
>> #ctx.notify_func = svn.client.svn_swig_py_notify_func
>> #ctx.notify_baton = update_callback
>>
>> ctx.notify_func2 = svn.client.svn_swig_py_notify_func
>> ctx.notify_baton2 = update_callback
>>
>> r = svn.core.svn_opt_revision_t()
>> r.kind = svn.core.svn_opt_revision_head
>>
>> try:
>>
>> result_revs = svn.client.svn_client_update3([wc], r,
>> svn.core.svn_depth_infinity, False, True, True, ctx)
>> for result_rev in result_revs:
>> print "result_rev: %d" % result_rev
>> print "done"
>>
>> except svn.core.SubversionException, e:
>> sys.stderr.write("Error (%d): %s\n" % (e.apr_err, e.message))
>> except Exception, e:
>> print "Unknown exception"
>> print e
>>
>>
>> I've also added some debug printfs to the underlying C functions
>> svn_swig_py_notify_func() and svn_swig_py_notify_func2() in swigutil_py.c to
>> check which function is being called and if the arguments coming into the C
>> function are the same as what is being reported by python.
>>
>
> Hi Lawrence,
>
> Have you considered trying out the new ctypes python bindings? I tried
> converting your script to use the new ctypes python bindings, and it
> seems to work fine. See the example below.
>
> from csvn.core import *
> import csvn.wc
> svn_cmdline_init("", csvn.core.stderr)
> wc = csvn.wc.WC("wc")
>
> def notify_func(obj):
> if obj.action == svn_wc_notify_update_update:
> print "U", obj.path
> elif obj.action == svn_wc_notify_update_completed:
> print "Completed", obj.path
>
> wc.checkout("http://svn.collab.net/repos/svn/trunk/www/")
> wc.update(revnum=32695)
> wc.set_notify_func(notify_func)
> wc.update(revnum=32696)
>
> The above example works on my machine, and outputs the following:
> U wc/faq.ja.html
> U wc
> Completed wc
>
> The ctypes python bindings are available at
> http://svn.collab.net/repos/svn/branches/ctypes-python-bindings/, and
> should work fine with Subversion 1.5.x. Hopefully they will be
> standard in Subversion 1.6.0, but we are still working on a few
> finishing touches.
No I haven't considered ctypes (wasn't even aware of it). Thanks for the
pointer. Is there other documentation than the README in the top level
branch dir that I should be referring to?
Is it anticipated that the ctypes interface will usurp the SWIG code? I
had kind of hoped to create a script that was backward compatible with
vanilla SVN back to 1.4.x hence the use of the SWIG API and not
something like PySVN, but I guess it's not disastrous if that's not the
case.
As a bit of background, I'm trying to create a script that allows me to
mirror a portion of a remote repo into my own development repo as a
vendor branch. The existing tools just don't seem to cut it for this use
case and hacking the svn C code seemed a bit extreme to get something
working relatively quickly. Given the trouble I'm having with the
bindings perhaps I should have just dived straight into writing a small
C app. I'm willing to be told otherwise if someone here has/knows such a
tool in which case my need for python bindings disappears :)
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by David James <ja...@gmail.com>.
On Sun, Aug 24, 2008 at 6:25 PM, Lawrence Stewart <ls...@room52.net> wrote:
> Lawrence Stewart wrote:
>>
>> Lawrence Stewart wrote:
>>>
>>> Hi all,
>>>
>>> I'm playing with the Python SWIG bindings for SVN 1.5.1 and have run into
>>> a brick wall. I want to update a working copy from a python script, and have
>>> a python function callback executed for each item updated. The python code
>>> below hopefully illustrates what I'm trying to do.
>>>
>>> def update_callback(notify, pool):
>>> print "in callback"
>>>
>>> def update(wc, rev):
>>> ctx = svn.client.svn_client_create_context()
>>> ctx.notify_func2 = update_callback
>>> r = svn.core.svn_opt_revision_t()
>>> r.kind = svn.core.svn_opt_revision_head
>>> result_rev = svn.client.svn_client_update(wc, r, True, ctx)
>>> print "result_rev: %d" % result_rev
>>>
>>> The update function does work and my working copy does get updated the
>>> HEAD, but my callback is never executed.
>>>
>>> I'm basing this code on the statement in
>>> http://svn.collab.net/svn-doxygen/group__Update.html that says if
>>> ctx->notify_func2 is not NULL it will be called for each item updated. This
>>> of course may not work or be applicable in Python land... someone please
>>> tell me if this is the case.
>>>
>>> I've tried countless variations of the above code and changing parameters
>>> to the callback, changing the way the callback is assigned to the context
>>> object (I tried following the way it was done in the client.py unit test
>>> file for a different callback type that is used) etc. etc. without success.
>>>
>>> I suspect my problem is that I just don't grok the translation that SWIG
>>> is doing from the C API up into Python and therein lies the problem.
>>>
>>> Any help in understanding what I'm doing wrong or how I can achieve
>>> something equivalent would be greatly appreciated.
>>
>>
>> Quick follow up... I just stumbled across this thread that describes what
>> I'm trying to do.
>>
>> http://osdir.com/ml/programming.swig/2003-06/msg00058.html
>>
>> It provided the missing clue I needed to make this work.
>>
>> Apologies for the noise.
>
>
> Ok, so I can now get the callback to fire, but the arguments passed to it
> look like random pointer junk and python coredumps somewhere within the
> svn.client.svn_client_update3() call after having executed the update and a
> few callbacks. Here's the new code:
>
>
>
> def update_callback(path, action, kind, mime_type, content_state,
> prop_state, revision):
>
> print "callback\taction: %d\trev: %u" % (action,revision)
>
>
>
> def update(wc, rev):
>
> ctx = svn.client.svn_client_ctx_t()
>
> #ctx.notify_func = svn.client.svn_swig_py_notify_func
> #ctx.notify_baton = update_callback
>
> ctx.notify_func2 = svn.client.svn_swig_py_notify_func
> ctx.notify_baton2 = update_callback
>
> r = svn.core.svn_opt_revision_t()
> r.kind = svn.core.svn_opt_revision_head
>
> try:
>
> result_revs = svn.client.svn_client_update3([wc], r,
> svn.core.svn_depth_infinity, False, True, True, ctx)
> for result_rev in result_revs:
> print "result_rev: %d" % result_rev
> print "done"
>
> except svn.core.SubversionException, e:
> sys.stderr.write("Error (%d): %s\n" % (e.apr_err, e.message))
> except Exception, e:
> print "Unknown exception"
> print e
>
>
> I've also added some debug printfs to the underlying C functions
> svn_swig_py_notify_func() and svn_swig_py_notify_func2() in swigutil_py.c to
> check which function is being called and if the arguments coming into the C
> function are the same as what is being reported by python.
>
Hi Lawrence,
Have you considered trying out the new ctypes python bindings? I tried
converting your script to use the new ctypes python bindings, and it
seems to work fine. See the example below.
from csvn.core import *
import csvn.wc
svn_cmdline_init("", csvn.core.stderr)
wc = csvn.wc.WC("wc")
def notify_func(obj):
if obj.action == svn_wc_notify_update_update:
print "U", obj.path
elif obj.action == svn_wc_notify_update_completed:
print "Completed", obj.path
wc.checkout("http://svn.collab.net/repos/svn/trunk/www/")
wc.update(revnum=32695)
wc.set_notify_func(notify_func)
wc.update(revnum=32696)
The above example works on my machine, and outputs the following:
U wc/faq.ja.html
U wc
Completed wc
The ctypes python bindings are available at
http://svn.collab.net/repos/svn/branches/ctypes-python-bindings/, and
should work fine with Subversion 1.5.x. Hopefully they will be
standard in Subversion 1.6.0, but we are still working on a few
finishing touches.
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks
Posted by Lawrence Stewart <ls...@room52.net>.
Lawrence Stewart wrote:
> Lawrence Stewart wrote:
>> Hi all,
>>
>> I'm playing with the Python SWIG bindings for SVN 1.5.1 and have run
>> into a brick wall. I want to update a working copy from a python
>> script, and have a python function callback executed for each item
>> updated. The python code below hopefully illustrates what I'm trying
>> to do.
>>
>> def update_callback(notify, pool):
>> print "in callback"
>>
>> def update(wc, rev):
>> ctx = svn.client.svn_client_create_context()
>> ctx.notify_func2 = update_callback
>> r = svn.core.svn_opt_revision_t()
>> r.kind = svn.core.svn_opt_revision_head
>> result_rev = svn.client.svn_client_update(wc, r, True, ctx)
>> print "result_rev: %d" % result_rev
>>
>> The update function does work and my working copy does get updated the
>> HEAD, but my callback is never executed.
>>
>> I'm basing this code on the statement in
>> http://svn.collab.net/svn-doxygen/group__Update.html that says if
>> ctx->notify_func2 is not NULL it will be called for each item updated.
>> This of course may not work or be applicable in Python land... someone
>> please tell me if this is the case.
>>
>> I've tried countless variations of the above code and changing
>> parameters to the callback, changing the way the callback is assigned
>> to the context object (I tried following the way it was done in the
>> client.py unit test file for a different callback type that is used)
>> etc. etc. without success.
>>
>> I suspect my problem is that I just don't grok the translation that
>> SWIG is doing from the C API up into Python and therein lies the problem.
>>
>> Any help in understanding what I'm doing wrong or how I can achieve
>> something equivalent would be greatly appreciated.
>
>
> Quick follow up... I just stumbled across this thread that describes
> what I'm trying to do.
>
> http://osdir.com/ml/programming.swig/2003-06/msg00058.html
>
> It provided the missing clue I needed to make this work.
>
> Apologies for the noise.
Ok, so I can now get the callback to fire, but the arguments passed to
it look like random pointer junk and python coredumps somewhere within
the svn.client.svn_client_update3() call after having executed the
update and a few callbacks. Here's the new code:
def update_callback(path, action, kind, mime_type, content_state,
prop_state, revision):
print "callback\taction: %d\trev: %u" % (action,revision)
def update(wc, rev):
ctx = svn.client.svn_client_ctx_t()
#ctx.notify_func = svn.client.svn_swig_py_notify_func
#ctx.notify_baton = update_callback
ctx.notify_func2 = svn.client.svn_swig_py_notify_func
ctx.notify_baton2 = update_callback
r = svn.core.svn_opt_revision_t()
r.kind = svn.core.svn_opt_revision_head
try:
result_revs = svn.client.svn_client_update3([wc], r,
svn.core.svn_depth_infinity, False, True, True, ctx)
for result_rev in result_revs:
print "result_rev: %d" % result_rev
print "done"
except svn.core.SubversionException, e:
sys.stderr.write("Error (%d): %s\n" % (e.apr_err, e.message))
except Exception, e:
print "Unknown exception"
print e
I've also added some debug printfs to the underlying C functions
svn_swig_py_notify_func() and svn_swig_py_notify_func2() in
swigutil_py.c to check which function is being called and if the
arguments coming into the C function are the same as what is being
reported by python.
Running the above code with a local working copy path and with my printf
enabled SWIG C code produces the following output on the console:
lstewart@lawrence1> ./submirror.py
result_rev: 1
svn_swig_py_notify_func path: action: 139968536 revision: 139968536
callback action: 139968536 rev: 139968536
svn_swig_py_notify_func path: (aW
action: 139943960 revision: 16778452
callback action: 139943960 rev: 16778452
svn_swig_py_notify_func path: 3W
action: 139792408 revision: 16778452
callback action: 139792408 rev: 16778452
svn_swig_py_notify_func path: U
action: 139792408 revision: 0
Segmentation fault (core dumped)
The lines starting with "svn_swig_py_notify_func" are being printed from
the C code, the others are coming form the python callback. Running GDB
on the python core file and getting a backtrace results in:
(gdb) bt
#0 0x28255ad9 in strlen () from /lib/libc.so.7
#1 0x080c8847 in PyModule_AddIntConstant ()
#2 0x080c8269 in PyModule_AddIntConstant ()
#3 0x080c8466 in PyModule_AddIntConstant ()
#4 0x080c8944 in PyModule_AddIntConstant ()
#5 0x0805b2e7 in PyObject_CallFunction ()
#6 0x282d2223 in svn_swig_py_notify_func (baton=0x8503aac,
path=0x8581af8 "\020\022U\b\v",
action=139792408, kind=139915304, mime_type=0x1 <Address 0x1 out of
bounds>,
content_state=svn_wc_notify_state_missing,
prop_state=svn_wc_notify_state_inapplicable, revision=2)
at
/usr/ports/devel/py-subversion/work/subversion-1.5.1/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c:2015
#7 0x28391265 in svn_client__update_internal () from
/usr/local/lib/libsvn_client-1.so.0
#8 0x28391458 in svn_client_update3 () from
/usr/local/lib/libsvn_client-1.so.0
#9 0x28928c45 in _wrap_svn_client_update3 () from
/usr/local/lib/python2.5/site-packages/libsvn/_client.so
#10 0x08059fe7 in PyObject_Call ()
#11 0x080acd4c in PyEval_CallObjectWithKeywords ()
#12 0x080ac5f4 in _PyBuiltin_Init ()
#13 0x080b18ee in PyEval_EvalFrameEx ()
#14 0x080b3109 in PyEval_EvalCodeEx ()
#15 0x080b15fa in PyEval_EvalFrameEx ()
#16 0x080b2425 in PyEval_EvalFrameEx ()
#17 0x080b3109 in PyEval_EvalCodeEx ()
#18 0x080b3257 in PyEval_EvalCode ()
#19 0x080ca8e6 in Py_CompileString ()
#20 0x080ca990 in PyRun_FileExFlags ()
#21 0x080cbe89 in PyRun_SimpleFileExFlags ()
#22 0x08056f91 in Py_Main ()
#23 0x08056455 in main ()
My current understanding is that instead of my current code, I should be
doing one of the following:
1.
ctx.notify_func2 = svn.client.svn_swig_py_notify_func2
ctx.notify_baton2 = update_callback2
Where svn.client.svn_swig_py_notify_func2 refers to the
svn_swig_py_notify_func2 C function in swigutil_py.c.
update_callback2 would have a signature applicable to the call made
in svn_swig_py_notify_func2 which to me looks like it should be:
update_callback2(notify, pool)
2.
ctx.notify_func = svn.client.svn_swig_py_notify_func
ctx.notify_baton = update_callback
Option 1 looks to be the correct way forward, but there is no
svn.client.svn_swig_py_notify_func2 exported from the python svn.client
module. Anyone have thoughts on why this is the case?
Option 2 looks like it should work but the callback is not executed at
all when I change the code accordingly.
Even though my current callback gets executed, I think my current code
is confusing things because I'm setting ctx.func2 to the C function that
is supposed to be called as the regular ctx.func which takes more
arguments... hence the pointer junk. That's just speculation on my
behalf though as I'm still slowly getting my head around how the SWIG
C->Python code is working.
Any help would be greatly appreciated.
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Help with Python SWIG bindings and callbacks [SOLVED]
Posted by Lawrence Stewart <ls...@room52.net>.
Lawrence Stewart wrote:
> Hi all,
>
> I'm playing with the Python SWIG bindings for SVN 1.5.1 and have run
> into a brick wall. I want to update a working copy from a python script,
> and have a python function callback executed for each item updated. The
> python code below hopefully illustrates what I'm trying to do.
>
> def update_callback(notify, pool):
> print "in callback"
>
> def update(wc, rev):
> ctx = svn.client.svn_client_create_context()
> ctx.notify_func2 = update_callback
> r = svn.core.svn_opt_revision_t()
> r.kind = svn.core.svn_opt_revision_head
> result_rev = svn.client.svn_client_update(wc, r, True, ctx)
> print "result_rev: %d" % result_rev
>
> The update function does work and my working copy does get updated the
> HEAD, but my callback is never executed.
>
> I'm basing this code on the statement in
> http://svn.collab.net/svn-doxygen/group__Update.html that says if
> ctx->notify_func2 is not NULL it will be called for each item updated.
> This of course may not work or be applicable in Python land... someone
> please tell me if this is the case.
>
> I've tried countless variations of the above code and changing
> parameters to the callback, changing the way the callback is assigned to
> the context object (I tried following the way it was done in the
> client.py unit test file for a different callback type that is used)
> etc. etc. without success.
>
> I suspect my problem is that I just don't grok the translation that SWIG
> is doing from the C API up into Python and therein lies the problem.
>
> Any help in understanding what I'm doing wrong or how I can achieve
> something equivalent would be greatly appreciated.
Quick follow up... I just stumbled across this thread that describes
what I'm trying to do.
http://osdir.com/ml/programming.swig/2003-06/msg00058.html
It provided the missing clue I needed to make this work.
Apologies for the noise.
Cheers,
Lawrence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org