You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David James <ja...@cs.toronto.edu> on 2008/06/10 15:26:29 UTC
Status of ctypes Subversion bindings
On Mon, Jun 9, 2008 at 5:01 AM, John Szakmeister <jo...@szakmeister.net> wrote:
> Hi! I assume that this email is reaching the same David James that is
> part of the Subversion project. I just wanted to ping you about the
> status of ctypesgen... and for that matter, csvn. It appears
> development has stalled (I'm not complaining... I certainly understand
> having demands for your time). I didn't know if this was a sign of
> "it's ready for prime time" or if you (and others) just didn't have
> the energy to finish it.
>
> I'd be interested in helping... I don't like SWIG much, and ctypes is
> a pain for complex APIs without something to help it. :-) BTW, is
> there a group that I should be posting this on? I didn't see anything
> on the Google Code page.
Hi John,
That's great news! We could certainly use the help.
Right now, the ctypes python bindings are in pretty good shape. The
low level bindings (which are similar to the SWIG bindings) are
complete and work really reliably and well.
We also have higher level python wrappers, which make it really easy
to use the python bindings.
The biggest work we need to do is figure out how to build the ctypes
bindings on Windows. It may be possible to just build the ctypes
python wrapper on Linux and copy it over to Windows and make a few
changes and it will work. This is because we are generating pure
Python bindings so in theory they could be platform independent. Note
that you will need Subversion as a DLL to use them in ctypes on
Windows, and that is a new feature in Subversion 1.5.
Currently the ctypes python bindings are on a branch and working
pretty well, but I'm not sure about where to go from here.
I see two options:
1) Merge the ctypes Python bindings to trunk and integrate them with
the Subversion build system
2) Launch ctypes bindings as a separate project.
Where do you think we should go from here?
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by John Szakmeister <jo...@szakmeister.net>.
On Tue, Jun 10, 2008 at 11:26 AM, David James <ja...@cs.toronto.edu> wrote:
[snip]
> Hi John,
>
> That's great news! We could certainly use the help.
Right now, I'm most interested in ctypesgen... it's be perfect to help
generate some bindings for another project I'm working. That said, I
am interested in something that's easier to use than the SWIG bindings
for Subversion. :-)
> Right now, the ctypes python bindings are in pretty good shape. The
> low level bindings (which are similar to the SWIG bindings) are
> complete and work really reliably and well.
Glad to hear it's working well!
> We also have higher level python wrappers, which make it really easy
> to use the python bindings.
>
> The biggest work we need to do is figure out how to build the ctypes
> bindings on Windows. It may be possible to just build the ctypes
> python wrapper on Linux and copy it over to Windows and make a few
> changes and it will work. This is because we are generating pure
> Python bindings so in theory they could be platform independent. Note
> that you will need Subversion as a DLL to use them in ctypes on
> Windows, and that is a new feature in Subversion 1.5.
I saw that it was using the C preprocessed output of gcc at some
point... I presume that's part of the problem when trying to make this
go under Windows? Are there any other issues? Where can I start
helping?
> Currently the ctypes python bindings are on a branch and working
> pretty well, but I'm not sure about where to go from here.
>
> I see two options:
> 1) Merge the ctypes Python bindings to trunk and integrate them with
> the Subversion build system
> 2) Launch ctypes bindings as a separate project.
>
> Where do you think we should go from here?
I'd prefer option 1.
BTW, sorry it took so long for me to respond... this got sucked
straight into my svn-dev folder and bypassed my inbox for some
reason... need to check my mail filter rules. :-)
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by Branko Čibej <br...@xbc.nu>.
Eric Gillespie wrote:
> I started testing csvn again, and found a problem.
> csvn/core/__init__.py calls svn_cmdline_init! That function does
> all kinds of things that are inappropriate for many programs
> (e.g. mucking with locale, mucking with stdout, printing to
> stderr, terminating the process). And it certainly shouldn't be
> called on import.
>
Clearly that function was meant to be used by (our) command-line tools,
so yes, bindings should never dream of calling it automatically.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by David James <ja...@cs.toronto.edu>.
On Sat, Jun 21, 2008 at 9:07 AM, David James <ja...@cs.toronto.edu> wrote:
> On Tue, Jun 17, 2008 at 7:41 PM, Eric Gillespie <ep...@pretzelnet.org> wrote:
>> I started testing csvn again, and found a problem.
>> csvn/core/__init__.py calls svn_cmdline_init! That function does
>> all kinds of things that are inappropriate for many programs
>> (e.g. mucking with locale, mucking with stdout, printing to
>> stderr, terminating the process). And it certainly shouldn't be
>> called on import.
>>
>>
>> Maybe we can add a csvn.SimpleInit function for high-level
>> consumers who don't want to think about; anyone else will have to
>> read the documentation to see what all they may have to
>> initialize, and how those things may fail.
>
> Great points. This will break compatibility with all scripts, but,
> fortunately, this is OK because the ctypes python bindings haven't
> been released yet. If everyone is OK with this idea I'll go ahead and
> implement this.
Hi Eric,
Per your suggestion, I got rid of the call to svn_cmdline_init in
r32005. I updated the example scripts in a few followup commits and
this should be complete as of r32009.
As of now, ctypes python bindings test suite should pass successfully
on at least the following platforms:
- Windows
- Linux 64-bit
- Linux 32-bit
- Mac OS X (32-bit)
To run the ctypes python bindings on Windows I generate the wrappers
for the bindings on Linux and then copy the python files to Windows.
This seems to work just fine.
On Windows and 64-bit Linux I went a bit further and ran the complete
svnmucc test suite on my example mucc.py. mucc.py passes all tests
now. (Note that I had to make a few tweaks to the svnmucc test suite
to get it to run on Windows, and to run outside of an svn build dir.)
It might make sense to integrate the mucc tests into the ctypes python
bindings test suite as they seem to be good at catching bugs.
I think we are getting closer to being able to merge the
ctypes-python-bindings to trunk alongside the SWIG bindings.
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by David James <ja...@cs.toronto.edu>.
On Tue, Jun 17, 2008 at 7:41 PM, Eric Gillespie <ep...@pretzelnet.org> wrote:
> I started testing csvn again, and found a problem.
> csvn/core/__init__.py calls svn_cmdline_init! That function does
> all kinds of things that are inappropriate for many programs
> (e.g. mucking with locale, mucking with stdout, printing to
> stderr, terminating the process). And it certainly shouldn't be
> called on import.
>
>
> Maybe we can add a csvn.SimpleInit function for high-level
> consumers who don't want to think about; anyone else will have to
> read the documentation to see what all they may have to
> initialize, and how those things may fail.
Great points. This will break compatibility with all scripts, but,
fortunately, this is OK because the ctypes python bindings haven't
been released yet. If everyone is OK with this idea I'll go ahead and
implement this.
On Tue, Jun 17, 2008 at 8:25 PM, Eric Gillespie <ep...@pretzelnet.org> wrote:
> Eric Gillespie <ep...@pretzelnet.org> writes:
>
>> I started testing csvn again, and found a problem.
>
> One more, and then I'm done for now.
>
>>>> revprops = csvn.core.apr_hash_make(pool)
>>>> csvn.core.apr_hash_set(revprops, 'svn:log', csvn.core.APR_HASH_KEY_STRING, csvn.core.svn_string_create('log message', pool))
>>>> val = csvn.core.apr_hash_get(revprops, 'key', csvn.core.APR_HASH_KEY_STRING)
>>>> val
> <ctypes.LP_c_void object at 0xf7aae6a4>
>>>> val_s = ctypes.cast(val, csvn.core.svn_string_t)
>
> Process Python segmentation fault (core dumped)
Looks like there's some typos in your script. (You put 'key' instead
of 'svn:log' in the apr_hash_get line, and you cast the val to a
svn_string_t instead of a pointer to a svn_string_t). When I fix these
typos your script works great:
import csvn.core
import csvn.types
import ctypes
pool = csvn.core.Pool()
revprops = csvn.core.apr_hash_make(pool)
s = csvn.core.svn_string_create('log message', pool)
csvn.core.apr_hash_set(revprops, 'svn:log', csvn.core.APR_HASH_KEY_STRING, s)
val = csvn.core.apr_hash_get(revprops, 'svn:log', csvn.core.APR_HASH_KEY_STRING)
val_s = ctypes.cast(val, ctypes.POINTER(csvn.core.svn_string_t))
print val_s[0].data
>
> OK, so I probably did something wrong. Maybe I can get
> csvn.types.Hash to work, and then I can pare that down until I
> understand what to do with a hash.
>
>>>> revprops = csvn.types.Hash(csvn.core.svn_string_t, wrapper=csvn.types.SvnStringPtr)
>>>> revprops
> {}
>>>> revprops['svn:log'] = 'log message'
>>>> revprops['svn:log']
>
> Process Python segmentation fault (core dumped)
You want a pointer to an svn_string_t here too:
revprops = csvn.types.Hash(ctypes.POINTER(csvn.core.svn_string_t),
wrapper=csvn.types.SvnStringPtr)
revprops['svn:log'] = 'log message'
print revprops['svn:log']
>
> Well, maybe guessing from uses of Hash in the high-level binding
> was wrong and I don't want the wrapper= parameter.
>
>>>> revprops = csvn.types.Hash(csvn.core.svn_string_t)
>>>> revprops['svn:log'] = 'log message'
>>>> revprops['svn:log']
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> File "/usr/local/google/home/epg/work/svn/ctypes-python-bindings6/csvn/types.py", line 59, in __getitem__
> raise KeyError(key)
> KeyError: 'svn:log'
>>>> revprops
>
> Process Python segmentation fault (core dumped)
>
Here's a working version of that script too:
revprops = csvn.types.Hash(ctypes.POINTER(csvn.core.svn_string_t))
s = csvn.core.svn_string_create('log message', pool)
revprops['svn:log'] = s
print revprops['svn:log'][0].data
Cheers,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> I started testing csvn again, and found a problem.
One more, and then I'm done for now.
>>> revprops = csvn.core.apr_hash_make(pool)
>>> csvn.core.apr_hash_set(revprops, 'svn:log', csvn.core.APR_HASH_KEY_STRING, csvn.core.svn_string_create('log message', pool))
>>> val = csvn.core.apr_hash_get(revprops, 'key', csvn.core.APR_HASH_KEY_STRING)
>>> val
<ctypes.LP_c_void object at 0xf7aae6a4>
>>> val_s = ctypes.cast(val, csvn.core.svn_string_t)
Process Python segmentation fault (core dumped)
OK, so I probably did something wrong. Maybe I can get
csvn.types.Hash to work, and then I can pare that down until I
understand what to do with a hash.
>>> revprops = csvn.types.Hash(csvn.core.svn_string_t, wrapper=csvn.types.SvnStringPtr)
>>> revprops
{}
>>> revprops['svn:log'] = 'log message'
>>> revprops['svn:log']
Process Python segmentation fault (core dumped)
Well, maybe guessing from uses of Hash in the high-level binding
was wrong and I don't want the wrapper= parameter.
>>> revprops = csvn.types.Hash(csvn.core.svn_string_t)
>>> revprops['svn:log'] = 'log message'
>>> revprops['svn:log']
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/local/google/home/epg/work/svn/ctypes-python-bindings6/csvn/types.py", line 59, in __getitem__
raise KeyError(key)
KeyError: 'svn:log'
>>> revprops
Process Python segmentation fault (core dumped)
Help? :)
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by Eric Gillespie <ep...@pretzelnet.org>.
I started testing csvn again, and found a problem.
csvn/core/__init__.py calls svn_cmdline_init! That function does
all kinds of things that are inappropriate for many programs
(e.g. mucking with locale, mucking with stdout, printing to
stderr, terminating the process). And it certainly shouldn't be
called on import.
I don't think calling that automatically is even acceptable for
the high-level binding. Who says my quick hack script has a
controlling tty? And who are you to terminate my process? :)
Maybe we can add a csvn.SimpleInit function for high-level
consumers who don't want to think about; anyone else will have to
read the documentation to see what all they may have to
initialize, and how those things may fail.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by David Glasser <gl...@davidglasser.net>.
On Fri, Jun 13, 2008 at 4:46 PM, Sage La Torra <sa...@gmail.com> wrote:
> On Wed, Jun 11, 2008 at 2:16 PM, David Glasser <gl...@davidglasser.net>
> wrote:
>>
>> On Tue, Jun 10, 2008 at 8:26 AM, David James <ja...@cs.toronto.edu> wrote:
>> > On Mon, Jun 9, 2008 at 5:01 AM, John Szakmeister <jo...@szakmeister.net>
>> > wrote:
>> >> Hi! I assume that this email is reaching the same David James that is
>> >> part of the Subversion project. I just wanted to ping you about the
>> >> status of ctypesgen... and for that matter, csvn. It appears
>> >> development has stalled (I'm not complaining... I certainly understand
>> >> having demands for your time). I didn't know if this was a sign of
>> >> "it's ready for prime time" or if you (and others) just didn't have
>> >> the energy to finish it.
>> >>
>> >> I'd be interested in helping... I don't like SWIG much, and ctypes is
>> >> a pain for complex APIs without something to help it. :-) BTW, is
>> >> there a group that I should be posting this on? I didn't see anything
>> >> on the Google Code page.
>> >
>> > Hi John,
>> >
>> > That's great news! We could certainly use the help.
>> >
>> > Right now, the ctypes python bindings are in pretty good shape. The
>> > low level bindings (which are similar to the SWIG bindings) are
>> > complete and work really reliably and well.
>> >
>> > We also have higher level python wrappers, which make it really easy
>> > to use the python bindings.
>> >
>> > The biggest work we need to do is figure out how to build the ctypes
>> > bindings on Windows. It may be possible to just build the ctypes
>> > python wrapper on Linux and copy it over to Windows and make a few
>> > changes and it will work. This is because we are generating pure
>> > Python bindings so in theory they could be platform independent. Note
>> > that you will need Subversion as a DLL to use them in ctypes on
>> > Windows, and that is a new feature in Subversion 1.5.
>> >
>> > Currently the ctypes python bindings are on a branch and working
>> > pretty well, but I'm not sure about where to go from here.
>> >
>> > I see two options:
>> > 1) Merge the ctypes Python bindings to trunk and integrate them with
>> > the Subversion build system
>> > 2) Launch ctypes bindings as a separate project.
>> >
>> > Where do you think we should go from here?
>>
>> I think we should ship the low-level, automatically-generated ctypes
>> bindings with Subversion starting with 1.6, and launch the high-level
>> more usable library built on top of that as a separate project.
>
> +1. I think that getting the full ctypes version into trunk will meet with
> some resistance, but this approach gets all the code out there and gets the
> most important part into trunk.
>
> Hopefully I'll have time to start coding again, getting my Master's
> seriously sidetracked development. Once I get set up at my new job I'll
> hopefully have a life again, and I can track down the few remaining issues.
Awesome! It would be great to see you having time to help out with
ctypes and svn again.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by Sage La Torra <sa...@gmail.com>.
On Wed, Jun 11, 2008 at 2:16 PM, David Glasser <gl...@davidglasser.net>
wrote:
> On Tue, Jun 10, 2008 at 8:26 AM, David James <ja...@cs.toronto.edu> wrote:
> > On Mon, Jun 9, 2008 at 5:01 AM, John Szakmeister <jo...@szakmeister.net>
> wrote:
> >> Hi! I assume that this email is reaching the same David James that is
> >> part of the Subversion project. I just wanted to ping you about the
> >> status of ctypesgen... and for that matter, csvn. It appears
> >> development has stalled (I'm not complaining... I certainly understand
> >> having demands for your time). I didn't know if this was a sign of
> >> "it's ready for prime time" or if you (and others) just didn't have
> >> the energy to finish it.
> >>
> >> I'd be interested in helping... I don't like SWIG much, and ctypes is
> >> a pain for complex APIs without something to help it. :-) BTW, is
> >> there a group that I should be posting this on? I didn't see anything
> >> on the Google Code page.
> >
> > Hi John,
> >
> > That's great news! We could certainly use the help.
> >
> > Right now, the ctypes python bindings are in pretty good shape. The
> > low level bindings (which are similar to the SWIG bindings) are
> > complete and work really reliably and well.
> >
> > We also have higher level python wrappers, which make it really easy
> > to use the python bindings.
> >
> > The biggest work we need to do is figure out how to build the ctypes
> > bindings on Windows. It may be possible to just build the ctypes
> > python wrapper on Linux and copy it over to Windows and make a few
> > changes and it will work. This is because we are generating pure
> > Python bindings so in theory they could be platform independent. Note
> > that you will need Subversion as a DLL to use them in ctypes on
> > Windows, and that is a new feature in Subversion 1.5.
> >
> > Currently the ctypes python bindings are on a branch and working
> > pretty well, but I'm not sure about where to go from here.
> >
> > I see two options:
> > 1) Merge the ctypes Python bindings to trunk and integrate them with
> > the Subversion build system
> > 2) Launch ctypes bindings as a separate project.
> >
> > Where do you think we should go from here?
>
> I think we should ship the low-level, automatically-generated ctypes
> bindings with Subversion starting with 1.6, and launch the high-level
> more usable library built on top of that as a separate project.
+1. I think that getting the full ctypes version into trunk will meet with
some resistance, but this approach gets all the code out there and gets the
most important part into trunk.
Hopefully I'll have time to start coding again, getting my Master's
seriously sidetracked development. Once I get set up at my new job I'll
hopefully have a life again, and I can track down the few remaining issues.
Sage
>
> That way, high-level API design can be decoupled from Subversion's
> release cycle, whereas the completely automated
> make-the-bindings-match-C step can guarantee that you always have
> immediate access to all new APIs.
>
> --dave
>
>
> --
> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
Re: Status of ctypes Subversion bindings
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"David Glasser" <gl...@davidglasser.net> writes:
> I think we should ship the low-level, automatically-generated ctypes
> bindings with Subversion starting with 1.6, and launch the high-level
> more usable library built on top of that as a separate project.
Along-side swig-py, which will then be marked deprecated, right?
Because csvn is drastically different.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by David Glasser <gl...@davidglasser.net>.
On Tue, Jun 10, 2008 at 8:26 AM, David James <ja...@cs.toronto.edu> wrote:
> On Mon, Jun 9, 2008 at 5:01 AM, John Szakmeister <jo...@szakmeister.net> wrote:
>> Hi! I assume that this email is reaching the same David James that is
>> part of the Subversion project. I just wanted to ping you about the
>> status of ctypesgen... and for that matter, csvn. It appears
>> development has stalled (I'm not complaining... I certainly understand
>> having demands for your time). I didn't know if this was a sign of
>> "it's ready for prime time" or if you (and others) just didn't have
>> the energy to finish it.
>>
>> I'd be interested in helping... I don't like SWIG much, and ctypes is
>> a pain for complex APIs without something to help it. :-) BTW, is
>> there a group that I should be posting this on? I didn't see anything
>> on the Google Code page.
>
> Hi John,
>
> That's great news! We could certainly use the help.
>
> Right now, the ctypes python bindings are in pretty good shape. The
> low level bindings (which are similar to the SWIG bindings) are
> complete and work really reliably and well.
>
> We also have higher level python wrappers, which make it really easy
> to use the python bindings.
>
> The biggest work we need to do is figure out how to build the ctypes
> bindings on Windows. It may be possible to just build the ctypes
> python wrapper on Linux and copy it over to Windows and make a few
> changes and it will work. This is because we are generating pure
> Python bindings so in theory they could be platform independent. Note
> that you will need Subversion as a DLL to use them in ctypes on
> Windows, and that is a new feature in Subversion 1.5.
>
> Currently the ctypes python bindings are on a branch and working
> pretty well, but I'm not sure about where to go from here.
>
> I see two options:
> 1) Merge the ctypes Python bindings to trunk and integrate them with
> the Subversion build system
> 2) Launch ctypes bindings as a separate project.
>
> Where do you think we should go from here?
I think we should ship the low-level, automatically-generated ctypes
bindings with Subversion starting with 1.6, and launch the high-level
more usable library built on top of that as a separate project.
That way, high-level API design can be decoupled from Subversion's
release cycle, whereas the completely automated
make-the-bindings-match-C step can guarantee that you always have
immediate access to all new APIs.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Status of ctypes Subversion bindings
Posted by Arfrever Frehtes Taifersar Arahesis <ar...@gmail.com>.
2008-06-10 17:26:29 David James napisał(a):
> I see two options:
> 1) Merge the ctypes Python bindings to trunk and integrate them with
> the Subversion build system
> 2) Launch ctypes bindings as a separate project.
>
> Where do you think we should go from here?
I would prefer option #1.
--
Arfrever Frehtes Taifersar Arahesis