You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David James <ja...@gmail.com> on 2005/08/02 04:12:54 UTC
Bugs in Subversion's SWIG 1.3.21 support
On my machine (Linux 2.4, using Subversion trunk or 1.2.1), the
"svnlook.py" script crashes if you run it on a nonempty repository
using SWIG 1.3.21. Here's a crash recipe (to be run from within the
tools/examples subdirectory):
svnadmin create repo
python putfile.py putfile.py repo
python svnlook.py repo
This SWIG 1.3.21 bug also causes the Trac test suite to crash with
both Subversion 1.2.1 and Subversion trunk.
(NOTE: This is a bug that affects SWIG 1.3.20 and SWIG 1.3.21 only.
Other supported versions of SWIG, including SWIG 1.3.19, SWIG 1.3.24,
and SWIG 1.3.25 are unaffected.)
After investigating this problem, I narrowed it down to a problem
inside the make_pointer function in swigutil_py.c:
static PyObject *make_pointer(const char *typename, void *ptr)
{
/* ### cache the swig_type_info at some point? */
return SWIG_NewPointerObj(ptr, SWIG_TypeQuery(typename), 0);
}
The problem: SWIG 1.3.21 isn't that great at TypeQuery lookups. For
example, if you ask for "svn_fs_root_t *", SWIG_TypeQuery returns
NULL. When you pass the NULL pointer into SWIG_NewPointerObj, it
segfaults.
Do other people also notice this problem? This looks like it's a bug
in SWIG, and it'll be difficult to work around. If other people also
notice this bug, I'd suggest we take one of three options:
1) Find some way to make Subversion and SWIG 1.3.20 and SWIG 1.3.21's
runtime lookup system work well together.
2) Avoid runtime type lookups altogether. Redesign Subversion's SWIG
library to use compile-time lookups instead.
3) Drop support for SWIG 1.3.20 and 1.3.21.
Opinions?
Cheers,
David
P.S. Before testing with SWIG 1.3.21, be careful to delete all
Python/SWIG-related files so that you don't accidentally test the
wrong version of SWIG. The 'make clean' rule in Subversion does not do
the job, because it does not delete Python/SWIG related files
properly.
Here is the build process I used to build and install SWIG:
./autogen.sh && ./configure --enable-static && make && make install &&
make runtime && make install-runtime
The "--enable-static" flag and the runtime installation targets are
essential with early versions of SWIG, but are not necessary with SWIG
1.3.24 and up.
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by John Lenz <le...@cs.wisc.edu>.
On Sun, August 7, 2005 4:23 pm, David Summers said:
> I'm trying out SWIG 1.3.25 on RH7.3, RH8, RH9, RHEL3, and RHEL4 to try to
> get rid of more package dependencies.
>
> So far on RHEL3 and RHEL4, no problems.
>
> I'm just now trying RH9 which uses python 2.2.2 and when I try the
> python -c "from svn import client", I get the following error:
>
> Traceback (most recent call last):
> File "<string>", line 1, in ?
> File "/usr/lib/svn-python/svn/client.py", line 19, in ?
> File "/usr/lib/svn-python/libsvn/client.py", line 5, in ?
> ImportError: /usr/lib/python2.2/site-packages/libsvn/_client.so: undefined
> symbol: SWIG_TypeQuery
>
>
> Any ideas what might be causing the undefined symbol? Anything I can do
> do to help debug it more? I'm using subversion trunk as of r15616.
In 1.3.25, SWIG_TypeQuery is declared static and included inline in every
file SWIG generates, so this error is very wierd. In 1.3.25, there is
absolutly no linking of library functions.
I would make sure that the SWIG you are building with is the correct one,
and that the paths are all set up... if you have multiple versions of
SWIG. Also note that SWIG needs the correct path to the library files
(SWIG_LIB) so you could be using the latest binary with old library
files... RH9 might include a copy of SWIG and that library path is above
the others in the search path. Check out the preinst-swig.sh file inside
the SWIG source, for how to run SWIG correctly without installing it.
John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by David Summers <da...@summersoft.fay.ar.us>.
On Thu, 4 Aug 2005, John Lenz wrote:
> On Mon, August 1, 2005 11:12 pm, David James said:
>
>> The problem: SWIG 1.3.21 isn't that great at TypeQuery lookups. For
>> example, if you ask for "svn_fs_root_t *", SWIG_TypeQuery returns
>> NULL. When you pass the NULL pointer into SWIG_NewPointerObj, it
>> segfaults.
>
> Thats correct. The older TypeQuerys had some problems...
>
>>
>> Do other people also notice this problem? This looks like it's a bug
>> in SWIG, and it'll be difficult to work around. If other people also
>> notice this bug, I'd suggest we take one of three options:
>> 1) Find some way to make Subversion and SWIG 1.3.20 and SWIG 1.3.21's
>> runtime lookup system work well together.
>
> The biggest problem with this is that before 1.3.23 or so, the runtime
> type stuff was a huge mess. It was using tons of hacks, the code was very
> difficult to follow, etc. With 1.3.25 for sure, the run time type stuff,
> while it didn't change much in functionality, got a lot easier to
> understand and the code is much more straight forward.
>
> Thus, trying to fix the 1.3.20 and 1.3.21 TypeQuery would be a lot of
> work. (well, it might be a small fix, but the problem is figuring out why
> TypeQuery isn't searching all the types...)
>
>> 2) Avoid runtime type lookups altogether. Redesign Subversion's SWIG
>> library to use compile-time lookups instead.
>
> I'm not all that familiar with the subversion sources and how difficult it
> would be, but I would say this would be the best solution. Now if it can
> be done with minimal support....?
>
>> 3) Drop support for SWIG 1.3.20 and 1.3.21.
>
> The other solution is what I have been talking about for a while now... as
> of 1.3.25, SWIG is a build time only tool, and the output from SWIG can be
> compiled on any computer. Think lex and yacc... SWIG now functions just
> like them.
>
> You can read the archives for some discussion on it, but the idea is that
> only the developers would need to have SWIG installed... you could include
> the build output from SWIG directly in the released tarballs. I find this
> idea similar to the reason neon and apr are included in the tarballs.
>
> If you stick the output .cxx files from SWIG into the svn repository, then
> only when a header or .i file changes will SWIG get invoked. I attempted
> to do this at one point, but the Makefile and python scripts to generate
> Makefiles were confusing so I didn't go that deep into it.
>
> The other advantage is that the developers can stay current with SWIG
> releases, geting all the bug fixes and such...
>
>> P.S. Before testing with SWIG 1.3.21, be careful to delete all
>> Python/SWIG-related files so that you don't accidentally test the
>> wrong version of SWIG. The 'make clean' rule in Subversion does not do
>> the job, because it does not delete Python/SWIG related files
>> properly.
>>
>> Here is the build process I used to build and install SWIG:
>> ./autogen.sh && ./configure --enable-static && make && make install &&
>> make runtime && make install-runtime
>>
>> The "--enable-static" flag and the runtime installation targets are
>> essential with early versions of SWIG, but are not necessary with SWIG
>> 1.3.24 and up.
>
> Yeah, that was one small piece of the mess of that runtime stuff....
>
I'm trying out SWIG 1.3.25 on RH7.3, RH8, RH9, RHEL3, and RHEL4 to try to
get rid of more package dependencies.
So far on RHEL3 and RHEL4, no problems.
I'm just now trying RH9 which uses python 2.2.2 and when I try the
python -c "from svn import client", I get the following error:
Traceback (most recent call last):
File "<string>", line 1, in ?
File "/usr/lib/svn-python/svn/client.py", line 19, in ?
File "/usr/lib/svn-python/libsvn/client.py", line 5, in ?
ImportError: /usr/lib/python2.2/site-packages/libsvn/_client.so: undefined
symbol: SWIG_TypeQuery
Any ideas what might be causing the undefined symbol? Anything I can do
do to help debug it more? I'm using subversion trunk as of r15616.
It works fine on subversion with swig 1.3.19 which I've been using up
until now.
Thanks!
--
David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint = 0B44 B118 85CC F4EC 7021 1ED4 1516 5B78 E320 2001
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by David Summers <da...@summersoft.fay.ar.us>.
Never mind, sorry to bother.
I rebuilt and things worked this time, I guess I did something in the
wrong order last time.
--
David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint = 0B44 B118 85CC F4EC 7021 1ED4 1516 5B78 E320 2001
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by David James <ja...@gmail.com>.
On 8/6/05, John Lenz <le...@cs.wisc.edu> wrote:
> > Perhaps it would be a good idea for us to run SWIG when users type
> > "autogen.sh"? (The SWIG bindings code would only be generated if SWIG
> > 1.3.25 is installed). We already include the generated output of
> > "autogen.sh" in the release tarballs.
>
> http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=97162
>
> http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=95920
>
> and the other messages on those threads
>
> and the older rather long thread
>
> http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=86430
>
> Some of those threads even have patches I wrote... probably don't work
> anymore but it is something you can look at.
>
> I don't presume to speek for the developers, but the last time this came
> up they seemed to like the idea... but didn't want to do anything prior to
> the release.
>
> Just to restate: all the files generated from SWIG 1.3.25 can be included
> directly and compiled anywhere... they have absolutly no SWIG
> dependencies. If other code needs access to the runtime type system (like
> subversion), you just need to run SWIG to generate the header file, and
> then that header file is also stand-alone. For more information, you can
> read the Modules chapter
>
> http://www.swig.org/Doc1.3/Modules.html
>
> basicly, just run
> swig -python -external-runtime <filename>
> which I think subversion is already doing... it just isn't doing it before
> the build process.
Thanks for your help, John! I've updated the
python-bindings-improvements branch in r15620 to follow your suggested
process. ./autogen.sh now compiles all SWIG files into standalone C
files if you run it with the --release flag. Users of the bindings
won't need to have SWIG installed, if they download a release version
of Subversion.
Cheers,
David
P.S. The python-bindings-improvements branch now requires SWIG 1.3.24.
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by John Lenz <le...@cs.wisc.edu>.
On Fri, August 5, 2005 8:27 am, David James said:
> This would be a huge improvement! I'm eager to get the latest bug
> fixes and improvements from the SWIG team, but I don't want to add
> 1.3.25 as a dependency. If we can treat SWIG like yacc or lex, then we
> can just bundle the generated wrapper files with the release tarball,
> and users won't even need SWIG, let alone SWIG 1.3.25.
>
> I think that this would be a great feature for Subversion 1.3.x. Can
> you explain what changes we'll need to make to our build system? If
> the other Subversion developers are also in support of the change, I
> can do the build system work.
>
> Perhaps it would be a good idea for us to run SWIG when users type
> "autogen.sh"? (The SWIG bindings code would only be generated if SWIG
> 1.3.25 is installed). We already include the generated output of
> "autogen.sh" in the release tarballs.
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=97162
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=95920
and the other messages on those threads
and the older rather long thread
http://subversion.tigris.org/servlets/ReadMsg?listName=dev&msgNo=86430
Some of those threads even have patches I wrote... probably don't work
anymore but it is something you can look at.
I don't presume to speek for the developers, but the last time this came
up they seemed to like the idea... but didn't want to do anything prior to
the release.
Just to restate: all the files generated from SWIG 1.3.25 can be included
directly and compiled anywhere... they have absolutly no SWIG
dependencies. If other code needs access to the runtime type system (like
subversion), you just need to run SWIG to generate the header file, and
then that header file is also stand-alone. For more information, you can
read the Modules chapter
http://www.swig.org/Doc1.3/Modules.html
basicly, just run
swig -python -external-runtime <filename>
which I think subversion is already doing... it just isn't doing it before
the build process.
Hope that helps,
John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by David James <ja...@gmail.com>.
> > 3) Drop support for SWIG 1.3.20 and 1.3.21.
>
> The other solution is what I have been talking about for a while now... as
> of 1.3.25, SWIG is a build time only tool, and the output from SWIG can be
> compiled on any computer. Think lex and yacc... SWIG now functions just
> like them.
>
> You can read the archives for some discussion on it, but the idea is that
> only the developers would need to have SWIG installed... you could include
> the build output from SWIG directly in the released tarballs. I find this
> idea similar to the reason neon and apr are included in the tarballs.
>
> If you stick the output .cxx files from SWIG into the svn repository, then
> only when a header or .i file changes will SWIG get invoked. I attempted
> to do this at one point, but the Makefile and python scripts to generate
> Makefiles were confusing so I didn't go that deep into it.
>
> The other advantage is that the developers can stay current with SWIG
> releases, geting all the bug fixes and such...
This would be a huge improvement! I'm eager to get the latest bug
fixes and improvements from the SWIG team, but I don't want to add
1.3.25 as a dependency. If we can treat SWIG like yacc or lex, then we
can just bundle the generated wrapper files with the release tarball,
and users won't even need SWIG, let alone SWIG 1.3.25.
I think that this would be a great feature for Subversion 1.3.x. Can
you explain what changes we'll need to make to our build system? If
the other Subversion developers are also in support of the change, I
can do the build system work.
Perhaps it would be a good idea for us to run SWIG when users type
"autogen.sh"? (The SWIG bindings code would only be generated if SWIG
1.3.25 is installed). We already include the generated output of
"autogen.sh" in the release tarballs.
> > 1) Find some way to make Subversion and SWIG 1.3.20 and SWIG 1.3.21's
> > runtime lookup system work well together.
> The biggest problem with this is that before 1.3.23 or so, the runtime
> type stuff was a huge mess. It was using tons of hacks, the code was very
> difficult to follow, etc. With 1.3.25 for sure, the run time type stuff,
> while it didn't change much in functionality, got a lot easier to
> understand and the code is much more straight forward.
This isn't a problem if we run SWIG in "autogen.sh", and bundle the
generated "C" files in the tarball.
Thanks for your help!
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bugs in Subversion's SWIG 1.3.21 support
Posted by John Lenz <le...@cs.wisc.edu>.
On Mon, August 1, 2005 11:12 pm, David James said:
> The problem: SWIG 1.3.21 isn't that great at TypeQuery lookups. For
> example, if you ask for "svn_fs_root_t *", SWIG_TypeQuery returns
> NULL. When you pass the NULL pointer into SWIG_NewPointerObj, it
> segfaults.
Thats correct. The older TypeQuerys had some problems...
>
> Do other people also notice this problem? This looks like it's a bug
> in SWIG, and it'll be difficult to work around. If other people also
> notice this bug, I'd suggest we take one of three options:
> 1) Find some way to make Subversion and SWIG 1.3.20 and SWIG 1.3.21's
> runtime lookup system work well together.
The biggest problem with this is that before 1.3.23 or so, the runtime
type stuff was a huge mess. It was using tons of hacks, the code was very
difficult to follow, etc. With 1.3.25 for sure, the run time type stuff,
while it didn't change much in functionality, got a lot easier to
understand and the code is much more straight forward.
Thus, trying to fix the 1.3.20 and 1.3.21 TypeQuery would be a lot of
work. (well, it might be a small fix, but the problem is figuring out why
TypeQuery isn't searching all the types...)
> 2) Avoid runtime type lookups altogether. Redesign Subversion's SWIG
> library to use compile-time lookups instead.
I'm not all that familiar with the subversion sources and how difficult it
would be, but I would say this would be the best solution. Now if it can
be done with minimal support....?
> 3) Drop support for SWIG 1.3.20 and 1.3.21.
The other solution is what I have been talking about for a while now... as
of 1.3.25, SWIG is a build time only tool, and the output from SWIG can be
compiled on any computer. Think lex and yacc... SWIG now functions just
like them.
You can read the archives for some discussion on it, but the idea is that
only the developers would need to have SWIG installed... you could include
the build output from SWIG directly in the released tarballs. I find this
idea similar to the reason neon and apr are included in the tarballs.
If you stick the output .cxx files from SWIG into the svn repository, then
only when a header or .i file changes will SWIG get invoked. I attempted
to do this at one point, but the Makefile and python scripts to generate
Makefiles were confusing so I didn't go that deep into it.
The other advantage is that the developers can stay current with SWIG
releases, geting all the bug fixes and such...
> P.S. Before testing with SWIG 1.3.21, be careful to delete all
> Python/SWIG-related files so that you don't accidentally test the
> wrong version of SWIG. The 'make clean' rule in Subversion does not do
> the job, because it does not delete Python/SWIG related files
> properly.
>
> Here is the build process I used to build and install SWIG:
> ./autogen.sh && ./configure --enable-static && make && make install &&
> make runtime && make install-runtime
>
> The "--enable-static" flag and the runtime installation targets are
> essential with early versions of SWIG, but are not necessary with SWIG
> 1.3.24 and up.
Yeah, that was one small piece of the mess of that runtime stuff....
John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org