You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@wandisco.com> on 2011/08/15 18:39:55 UTC

Ctypes-python Array class self-reference problem

For anyone interested: I'm trying to debug a memory problem in the
ctypes-python bindings Array class which models an apr_array_header_t
array.  I suppose this class may never have worked properly, as it gets
exercised very little within the bindings and had no unit test cases
until today.

Running within its test suite it mostly works, but there appears to be a
problem in making a deep copy of, or holding onto a reference to, each
element when a new array is initialized.

A simple demonstration in the Python interpreter:

$ (cd subversion/bindings/ctypes-python/ && python -i
test/setup_path.py)
>>> from csvn.core import *
>>> from csvn.types import Array
>>> a1 = Array(c_char_p, ['one'])
>>> a1
Array(['Array('])

The last line should have shown "Array(['one'])", but it printed garbage
from the Python interpreter's memory.  This is so bad that I wonder if
my expectations are wrong.

The attached patch contains various changes and additions that I tried
unsuccessfully, and it adds tests that all pass (for me) except for the
places where they extend an Array with another (or the same) array.
Initially I thought the problem was only with self-referencing, because
I saw the failures in expressions such as "a1.extend(a1)", but now I am
noticing the same effect more widely.

Any hints appreciated.

- Julian


Re: Ctypes-python Array class self-reference problem

Posted by Hyrum K Wright <hy...@wandisco.com>.
On Wed, Aug 17, 2011 at 7:58 AM, Julian Foad <ju...@wandisco.com> wrote:
>
> I have read up about ctypes and tried to understand in detail what's
> going on here but I've now put it aside as it seems too much of a time
> sink.

Welcome to the wonderful world of bindings maintenance!

-Hyrum


-- 

uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/

Re: Ctypes-python Array class self-reference problem

Posted by Julian Foad <ju...@wandisco.com>.
Just to follow up here, this isn't specifically a self-reference
problem.  I wonder if it's a problem of by-value vs. by-reference
semantics.

I have read up about ctypes and tried to understand in detail what's
going on here but I've now put it aside as it seems too much of a time
sink.

- Julian


On Tue, 2011-08-16 at 14:52 +0300, Daniel Shahaf wrote:
> I'm having trouble getting even 'Array(c_void_p)' to work.
> 
> It either outputs nothing and opens a new line with another >>> prompt,
> that has some functionality disabled (eg, prompt history), or complains
> >>> x = Array(c_void_p, [])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: abstract class
> .
> 
> 
> 
> Julian Foad wrote on Mon, Aug 15, 2011 at 17:39:55 +0100:
> > For anyone interested: I'm trying to debug a memory problem in the
> > ctypes-python bindings Array class which models an apr_array_header_t
> > array.  I suppose this class may never have worked properly, as it gets
> > exercised very little within the bindings and had no unit test cases
> > until today.
> > 
> > Running within its test suite it mostly works, but there appears to be a
> > problem in making a deep copy of, or holding onto a reference to, each
> > element when a new array is initialized.
> > 
> > A simple demonstration in the Python interpreter:
> > 
> > $ (cd subversion/bindings/ctypes-python/ && python -i
> > test/setup_path.py)
> > >>> from csvn.core import *
> > >>> from csvn.types import Array
> > >>> a1 = Array(c_char_p, ['one'])
> > >>> a1
> > Array(['Array('])
> > 
> > The last line should have shown "Array(['one'])", but it printed garbage
> > from the Python interpreter's memory.  This is so bad that I wonder if
> > my expectations are wrong.
> > 
> > The attached patch contains various changes and additions that I tried
> > unsuccessfully, and it adds tests that all pass (for me) except for the
> > places where they extend an Array with another (or the same) array.
> > Initially I thought the problem was only with self-referencing, because
> > I saw the failures in expressions such as "a1.extend(a1)", but now I am
> > noticing the same effect more widely.
> > 
> > Any hints appreciated.
> > 
> > - Julian
> > 
> 



Re: Ctypes-python Array class self-reference problem

Posted by Daniel Shahaf <da...@elego.de>.
I'm having trouble getting even 'Array(c_void_p)' to work.

It either outputs nothing and opens a new line with another >>> prompt,
that has some functionality disabled (eg, prompt history), or complains
>>> x = Array(c_void_p, [])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: abstract class
.



Julian Foad wrote on Mon, Aug 15, 2011 at 17:39:55 +0100:
> For anyone interested: I'm trying to debug a memory problem in the
> ctypes-python bindings Array class which models an apr_array_header_t
> array.  I suppose this class may never have worked properly, as it gets
> exercised very little within the bindings and had no unit test cases
> until today.
> 
> Running within its test suite it mostly works, but there appears to be a
> problem in making a deep copy of, or holding onto a reference to, each
> element when a new array is initialized.
> 
> A simple demonstration in the Python interpreter:
> 
> $ (cd subversion/bindings/ctypes-python/ && python -i
> test/setup_path.py)
> >>> from csvn.core import *
> >>> from csvn.types import Array
> >>> a1 = Array(c_char_p, ['one'])
> >>> a1
> Array(['Array('])
> 
> The last line should have shown "Array(['one'])", but it printed garbage
> from the Python interpreter's memory.  This is so bad that I wonder if
> my expectations are wrong.
> 
> The attached patch contains various changes and additions that I tried
> unsuccessfully, and it adds tests that all pass (for me) except for the
> places where they extend an Array with another (or the same) array.
> Initially I thought the problem was only with self-referencing, because
> I saw the failures in expressions such as "a1.extend(a1)", but now I am
> noticing the same effect more widely.
> 
> Any hints appreciated.
> 
> - Julian
>