You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David James <ja...@gmail.com> on 2005/06/29 02:33:20 UTC
Python Command-line Client Bindings
This summer, as part of the Google Summer of Code initiative, I'm
planning to create a port of the Subversion Command-Line client that
uses the Python/SWIG bindings. Once this new software is complete,
we'll be able to use the standard tests for the regular Subversion
Command-Line Client to identify bugs in the Python bindings.
The current state of the Python tests is glibly described in
tools/test-scripts/svntest-bindings.sh:
"Hey! My friends have got beautiful and shining unit tests,
but I have been left out in the cold. This is soooo unfair!"
Here's what I'm planning to do in the next few weeks:
- Refactor the C command-line client:
* Create a libsvn_cmdline library by extracting functions from
the main command-line client and moving them to the
subversion/libsvn_cmdline directory.
* Create new API header file for the libsvn_cmdline library
based on cl.h. I'm thinking of calling the new header file
svn_cmdline_cl.h.
* Simplify the svn_cmdline_cl.h file so that SWIG can have
an easier time parsing it. This means we'll have to change
functions declared as callbacks into vanilla function
definitions. The standard syntax isn't as compact,
but it's SWIG-compatible.
* Simplify the main function in the command-line client by
splitting it into several smaller functions. This refactoring
will make it easier for me to slowly convert the main function
from C into Python, piece by piece.
- Expand the functionality of the Python/SWIG bindings:
* Create new svn_cmdline module for the new libsvn_cmdline library
* Create new svn_diff module for the libsvn_diff library. This is
mostly a stub for now.
* Create new svn_apr module for the APR library. Create mappings
for any functionality that is used in the command-line client.
Two candidates right now: the apr_getopt.h and apr_allocator.h
header files.
* Create bindings for the libsvn_cmdline library
* Create new typemaps for types such as char **
- Refactor the Python/SWIG bindings:
* Move the FILE* typemap from core.i to a new file called 'file.i'
so that it can be reused in the new svn_cmdline bindings.
* Remove the svn_cmdline_init function from the svn_delta.i module.
This function has been added to the svn_cmdline module instead.
- Convert the C command-line client to Python, piece by piece.
I'll start with the main function. Then I'll move on to the
implementations of each basic Subversion command:
* add, blame, cat, checkout, cleanup, commit, copy, delete,
diff, export, help, import, info, list, lock, log, merge,
mkdir, move, propdel, propedit, propget, proplist, propset,
resolved, revert, status, switch, unlock, update
To help keep the implementation and maintenance of the new
functions simple, the C code for each function will be
directly transliterated into Python. I'll try to use the
exact same API calls and the exact same logic but in Python
syntax.
Along the way, I'll be sure to run into a few issues with
the C code and the SWIG bindings. Some of these issues have
already been identified above.
Questions:
- Is it OK to create the new libsvn_cmdline library? It will make my
command-line SWIG work this summer much easier.
- Is it possible to mark the libsvn_cmdline_cl.h API as experimental
and subject to change? I think that the API for the command-line
client could go through quite a few changes and improvements this
summer and I don't want to have to worry about revving the API yet.
- Is it OK to use malloc / free in the SWIG bindings? In some cases,
this seems to be the only solution because APR pools are not always
available in a typemap.
For those who haven't read my Python proposal, I've attached it below.
Feedback and advice is always welcome.
Project Title: Command-line Bindings for Python
----------------------------------------------------------------------
Synopsis
----------------------------------------------------------------------
Subversion is not just a version-control system. It is also a library.
The Subversion library officially supports five programming languages:
C, Perl, Python, Ruby, and Java. As Subversion is updated to fix bugs
and support new features, it is often difficult to know whether these
changes will cause problems in the various programming language
bindings. This problem of hidden bugs is particularly acute in the
Python/SWIG bindings because they do not have an automated test suite.
To help Subversion developers more quickly identify bugs in the Python
bindings, I will implement a clone of the standard command-line
client using the Python/SWIG bindings. This clone will allow the
existing test suite for the Subversion command-line client to also test
the Python command-line client. If this testing reveals bugs or missing
features in the underlying Python/SWIG bindings, the Subversion
developers will automatically be notified via the svn-breakage mailing
list.
----------------------------------------------------------------------
Benefits
----------------------------------------------------------------------
Benefits for Python Developers:
- Simple and consistent usage: Type the same commands into Python as
you would on the command-line.
- Because the new interface is implemented based on the existing
low-level library, high-level command-line calls and low-level
library calls can be intermixed during a single session.
- The existing high-performance Python/SWIG bindings now have proven
reliability, thanks to the extensive automated test suite for the
command-line client.
Benefits for Subversion Developers:
- Increased adoption of Subversion in the Python community.
- Automatic nightly test suite will notify developers if changes to the
Subversion code break the Python bindings.
- Upgraded Python bindings will be easy to maintain because they build
upon the framework established in the existing SWIG bindings (e.g.
Ruby, Perl, Python)
----------------------------------------------------------------------
Deliverables
----------------------------------------------------------------------
Code:
- 31 functions. Each implements a basic command
* add, blame, cat, checkout, cleanup, commit, copy, delete, diff,
export, help, import, info, list, lock, log, merge, mkdir, move,
propdel, propedit, propget, proplist, propset, resolved, revert,
status, switch, unlock, update
- 32 command-line options
* auto-props, config-dir, diff-cmd, diff3-cmd, dry-run, editor-cmd,
encoding, extensions, file, force, force-log, ignore-ancestry,
ignore-externals, incremental, limit, message, native-eol, new,
no-auth-cache, no-auto-props, no-diff-deleted, no-ignore,
no-unlock, non-interactive, non-recursive, notice-ancestry,
old, password, quiet, recursive, relocate, revision, revprop,
show-updates, stop-on-copy, strict, targets, username, verbose,
version
Testing:
- The test suite for the standard client will be adapted so that it can
test the Python client
----------------------------------------------------------------------
Implementation Plan
----------------------------------------------------------------------
1. Write a simple script which creates a Python command-line parser
based on the svn_cl__options and svn_cl__cmd_table structures from
main.c in subversion/clients/cmdline. The Python standard optparse
module will do most of the work, but we will also need to write
custom code to parse Subversion revision numbers and ranges.
2. Upgrade our script to dispatch each command to the appropriate
command-line client function. This initial prototype will provide
the full functionality of the Subversion client, but will only test
the surface functionality of SWIG.
3. Upgrade the Subversion automated test suite to test our new script
using the command-line test suite. Fix any errors that are found.
4. Replace each command-line C function in the Python command-line
client with an appropriate Python implementation. Each function
should be implemented, tested, and committed as a separate patch. If
adding a new function reveals a bug in the underlying SWIG/Python
library, these bugs should be reported to the Subversion development
list.
----------------------------------------------------------------------
Project Schedule
----------------------------------------------------------------------
This 2 month plan assumes that I will start work on this project
on June 25. The project will be complete by September 1, 2005
1. Planning and Approval (4 days)
- Send the technical details of my plan to Subversion developers
and solicit feedback
- Revise my plan as necessary to meet the needs of the Subversion
developers
- Apply to be a partial committer for the Python bindings
2. Initial Prototype (2 weeks)
- Write a simple script which creates a Python command-line parser
- Upgrade our script to dispatch each command to the appropriate C
function
- Upgrade the Subversion automated test suite to test our new script
4. Implementation, Documentation & Testing (7 weeks)
- Replace each command-line C function in the Python command-line
client with an appropriate Python implementation. Implement 5 or 6
API functions per week.
- Monitor Subversion developer list and fix issues as required
5. Project is complete
----------------------------------------------------------------------
Appendix A: Why stick with the existing Python/SWIG bindings?
----------------------------------------------------------------------
Subversion developers love SWIG because it saves them time. "Sharing
the core of the bindings implementation across languages", writes
Daniel Rall, "is powerful reuse." [1]
Nevertheless, the Python/SWIG bindings are not the only libraries which
offer access to the Subversion library. The PySVN and SvnCpp projects
are both written in C++, and they both offer documented and tested
bindings for the Subversion library. However, both sets of bindings
suffer from the same key problem: they only wrap a small subset of
Subversion's functionality, and they reimplement functionality which is
already available in the existing SWIG bindings for Subversion.
Maintaining the two sets of bindings in parallel would be too large a
task for the Subversion development team.
As a result of this situation, Python developers are forced to make a
difficult choice between ease of use and complete functionality.
According to Max Bowsher, "PySVN wraps much less of the API than the
SWIG-Python bindings do, but does it in a higher level (and documented)
way -- it's all about tradeoffs, really." [2] The situation with SvnCpp
is much the same as with PySVN, except for that SvnCpp does not
directly support Python.
In 2004, Ben Reser announced that, for Python, the "SWIG stuff is
pretty much done. You could write the OO layer entirely in Python" [3]
To a developer who considered reimplementing the bindings in Pyrex, Ben
Reser advised: "I think your time would be better spent working on
writing the OO layer on top of SWIG." [3]
Greg Stein also attests to the quality of the SWIG/Python bindings:
"I've been using the Python Bindings for years. Literally." [4]
In building a command-line client in Python, we will get an extensive
test suite and a rudimentary object-oriented interface to the
SWIG/Python bindings for free. While this interface will initially only
support the basic functionality of the Subversion client, we can in
future extend this interface to support additional functionality.
[1]: http://svn.haxx.se/dev/archive-2004-04/1044.shtml
[2]: http://svn.haxx.se/dev/archive-2005-02/0748.shtml
[3]: http://svn.haxx.se/dev/archive-2004-04/1395.shtml
[4]: http://svn.haxx.se/dev/archive-2004-05/0407.shtml
----------------------------------------------------------------------
Appendix B: Functionality Listing
----------------------------------------------------------------------
All documented commands of the command-line client will be supported.
That is, the following commands will be supported.
* add: Add files, directories, or symbolic links to your working
copy and schedule them for addition to the repository.
* blame: Show author and revision information in-line for the
specified files or URLs
* cat: Output the contents of the specified files or URLs
* checkout: Check out a working copy from a repository
* cleanup: Recursively clean up the working copy
* commit: Send changes from your working copy to the repository
* copy: Copy a file or directory in a working copy or in the
repository
* delete: Delete an item from a working copy or the repository
* diff: Display the differences between two paths
* export: Export a clean directory tree
* help: Describe the usage of this program or its subcommands
* import: Recursively commit a copy of PATH to URL
* info: Print information about PATHs
* list: List directory entries in the repository
* lock: Lock working copies paths or URLs in the repository, so
that no other user can commit changes to them.
* log: Display commit log messages
* merge: Apply the differences between two sources to a working
copy path
* mkdir: Create a new directory under version control
* move: Move a file or directory
* propdel: Remove a property from an item
* propedit: Edit the property of one or more items under version
control
* propget: Print the value of a property
* proplist: List all properties
* propset: Set PROPNAME to PROPVAL on files, directories, or
revisions
* resolved: Remove 'conflicted' state on working copy files or
directories
* revert: Undo all local edits
* status: Print the status of working copy files and directories
* switch: Update working copy to a different URL
* unlock: Unlock working copies paths or URLs
* update: Update your working copy
The following command-line options will be supported:
* auto-props: enable automatic properties
* config-dir: read user configuration files from directory ARG
* diff-cmd: use ARG as diff command
* diff3-cmd: use ARG as merge command
* dry-run: try operation but make no changes
* editor-cmd: use ARG as external editor
* encoding: treat value as being in specified charset
encoding
* extensions: pass ARG to --diff-cmd as options
* file: read data from specified file
* force-log: force validity of log message source
* force: force operation to run
* help: show help on a subcommand
* ignore-ancestry: ignore ancestry when calculating merges
* ignore-externals: ignore externals definitions
* incremental: give output suitable for concatenation
* limit: maximum number of log entries
* message: specify commit message ARG
* native-eol: use a different EOL marker than the standard
system marker for files with a native svn:eol-
style property. ARG may be one of 'LF', 'CR',
'CRLF'
* new: use ARG as the newer target
* no-auth-cache: do not cache authentication tokens
* no-auto-props: disable automatic properties
* no-diff-deleted: do not print differences for deleted files
* no-ignore: disregard default and svn:ignore property ignores
* no-unlock: don't unlock the targets
* non-interactive: do no interactive prompting
* non-recursive: operate on single directory only,
* notice-ancestry: notice ancestry when calculating differences
* old: use ARG as the older target
* password: specify a password
* quiet: print as little as possible,
* recursive: descend recursively
* relocate: relocate via URL-rewriting
* revision: a revision or a range of revisions
* revprop: operate on a revision property (use with -r)
* show-updates: display update information
* stop-on-copy: do not cross copies while traversing history
* strict: use strict semantics
* targets: pass contents of file ARG as additional args
* username: specify a username
* verbose: print extra information
* version: print client version info
* xml: output in XML
----------------------------------------------------------------------
Appendix C: Biography
----------------------------------------------------------------------
David James is an undergraduate Computer Science student at the
University of Toronto. In Fall 2004, David helped write Subversion
bindings for a Java-based academic groupware system. Since then, David
has been a regular contributor to the Subversion project, submitting
17 patches which were reviewed and accepted by Subversion developers.
David's contributions have made the Java and Ruby bindings easier to
compile, test, and install. Recently, David added Ruby support to the
automated nightly test-suite, so that Subversion developers can be
notified by email whenever a Ruby test fails. David is a partial
committer for the Ruby bindings.
At the University of Toronto, David has researched improved statistical
models for understanding natural language, earning three research
awards and a teaching assistantship in the process. As part of his
teaching assistantship, David taught Python to Computational
Linguistics graduate students.
For more information on David James, please see my resume:
http://www.cs.toronto.edu/~james/David_James_Resume_2005.html
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by David James <ja...@gmail.com>.
On 6/29/05, Garrett Rooney <ro...@electricjellyfish.net> wrote:
> What's the point of calling the command line client's main from python
> in the first place? Won't that just call through to the underlying
> svn_client_foo functions, bypassing all the python wrappers you intend
> to test by building this 'command line client in python'?
Yes, that's right. According to my proposal, "This initial prototype
will provide the full functionality of the Subversion client, but will
only test the surface functionality of SWIG." After that, I'll rewrite
the command-line client piece by piece, starting with the main
function.
Of course, this whole discussion is moot if you would prefer that I
take a different approach. See my other email for alternatives:
http://svn.haxx.se/dev/archive-2005-06/1074.shtml
Cheers,
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by David James <ja...@gmail.com>.
On 30 Jun 2005 09:59:54 -0500, kfogel@collab.net <kf...@collab.net> wrote:
> I think that's a good idea. I mean, the real point was always to get
> the Python bindIings thoroughly tested, and improved as a result. The
> Python command-line client was just a means to that end; if a better
> means is available, we should use that.
Great! I'll email the list when I have more details on my new plan.
Cheers,
David
>
> By the way, I've taken soc@ off the CC list. That mailing list is not
> for technical discussions, only for SoC administrivia and such.
>
> -Karl
>
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by kf...@collab.net.
David James <ja...@gmail.com> writes:
> Sounds good. I think that your suggested approach would be easier and
> could offer more benefits to the project as a whole. If I keep the
> tests relatively simple, it will leave more time for me to work on
> other aspects of the bindings, such as automatic pool management.
>
> Opinions?
I think that's a good idea. I mean, the real point was always to get
the Python bindings thoroughly tested, and improved as a result. The
Python command-line client was just a means to that end; if a better
means is available, we should use that.
Sorry for the severe whiplash your poor proposal must be suffering by
now, David :-). In retrospect, I wish that those of us reviewing the
SoC applications (most svn developers did not, though all were free
to) had consulted with Mike Pilato and other Python-bindings people
about the specifics of yours, so that we could have gotten here
sooner. Oh well, live and learn. Thanks for being so darn reasonable
about it all, though.
By the way, I've taken soc@ off the CC list. That mailing list is not
for technical discussions, only for SoC administrivia and such.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by David James <ja...@gmail.com>.
C. Michael Pilato wrote:
> David James writes:
> > C. Michael Pilato wrote:
> > > I have to say (and have said in a few private email already today)
> > > that I'm quite confused by the point of this whole effort.
> > Would you prefer if I took the more direct approach and just built
> > thorough tests for the Python bindings? I'm not married to the idea of
> > creating a command-line client in Python. :) In fact, it might make my
> > life easier to take the direct approach. What do you think?
> I would very much prefer such a thing. I've started down the road
> myself several times, but kept getting hung up on our build system.
> I'd like it very much if 'make swig-py' would create ready-to-install
> / ready-to-test modules linked against the C libraries still in the
> build tree. Upon that foundation could 'make swig-py-check' be built.
Sounds good. I think that your suggested approach would be easier and
could offer more benefits to the project as a whole. If I keep the
tests relatively simple, it will leave more time for me to work on
other aspects of the bindings, such as automatic pool management.
Opinions?
Cheers,
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by "C. Michael Pilato" <cm...@collab.net>.
David James <ja...@gmail.com> writes:
> C. Michael Pilato wrote:
> > I have to say (and have said in a few private email already today)
> > that I'm quite confused by the point of this whole effort.
> >
> > If the goal is to make the Python bindings testable, I think it's
> > overkill. The bindings are very thin; smaller test scripts would work
> > just fine. I mean, all you really need to test is that data can get
> > from Python to C and back again -- 98% of the work a Python bindings
> > cmdline client would do is down in the C libraries, which are already
> > well-covered by our test framework.
> Would you prefer if I took the more direct approach and just built
> thorough tests for the Python bindings? I'm not married to the idea of
> creating a command-line client in Python. :) In fact, it might make my
> life easier to take the direct approach. What do you think?
I would very much prefer such a thing. I've started down the road
myself several times, but kept getting hung up on our build system.
I'd like it very much if 'make swig-py' would create ready-to-install
/ ready-to-test modules linked against the C libraries still in the
build tree. Upon that foundation could 'make swig-py-check' be built.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by David James <ja...@gmail.com>.
C. Michael Pilato wrote:
> I have to say (and have said in a few private email already today)
> that I'm quite confused by the point of this whole effort.
>
> If the goal is to make the Python bindings testable, I think it's
> overkill. The bindings are very thin; smaller test scripts would work
> just fine. I mean, all you really need to test is that data can get
> from Python to C and back again -- 98% of the work a Python bindings
> cmdline client would do is down in the C libraries, which are already
> well-covered by our test framework.
Would you prefer if I took the more direct approach and just built
thorough tests for the Python bindings? I'm not married to the idea of
creating a command-line client in Python. :) In fact, it might make my
life easier to take the direct approach. What do you think?
Cheers,
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by "C. Michael Pilato" <cm...@collab.net>.
Garrett Rooney <ro...@electricjellyfish.net> writes:
> David James wrote:
>
> > Here's an example of a function (from main.c) which is difficult to
> > Swigify without using malloc/free.
> > int svn_cmdline_main (int argc, const char * const *argv)
> > (This is the 'main' function from clients/cmdline/main.c)
> > When you call the svn_cmdline_main function using Python strings, we
> > need to allocate memory to store C-compatible versions of these
> > strings. We can easily accomplish this using malloc/free. Allocating
> > a
> > brand new APR pool for this purpose would be very wasteful,
> > especially since the purpose of the svn_cmdline_main function
> > already allocates its own root pool. So, I think the best solution
> > here is to use malloc/free. What do you think?
>
> What's the point of calling the command line client's main from python
> in the first place? Won't that just call through to the underlying
> svn_client_foo functions, bypassing all the python wrappers you intend
> to test by building this 'command line client in python'?
I have to say (and have said in a few private email already today)
that I'm quite confused by the point of this whole effort.
If the goal is to make the Python bindings testable, I think it's
overkill. The bindings are very thin; smaller test scripts would work
just fine. I mean, all you really need to test is that data can get
from Python to C and back again -- 98% of the work a Python bindings
cmdline client would do is down in the C libraries, which are already
well-covered by our test framework.
If the goal if to find holes in our Python bindings coverage, I don't
think those of us that actually work on those bindings need the help.
With just a moderate knowledge of SWIG, one can visually scan our C
API and know which interfaces are and aren't going to be troublesome
for SWIG to deal with.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
David James wrote:
> Here's an example of a function (from main.c) which is difficult to
> Swigify without using malloc/free.
> int svn_cmdline_main (int argc, const char * const *argv)
> (This is the 'main' function from clients/cmdline/main.c)
>
> When you call the svn_cmdline_main function using Python strings, we
> need to allocate memory to store C-compatible versions of these
> strings. We can easily accomplish this using malloc/free. Allocating a
> brand new APR pool for this purpose would be very wasteful, especially
> since the purpose of the svn_cmdline_main function already allocates
> its own root pool. So, I think the best solution here is to use
> malloc/free. What do you think?
What's the point of calling the command line client's main from python
in the first place? Won't that just call through to the underlying
svn_client_foo functions, bypassing all the python wrappers you intend
to test by building this 'command line client in python'?
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by David James <ja...@gmail.com>.
Ben Collins-Sussman <su...@collab.net> wrote:
> Take a look at minimal_client.c . All a commandline-based client
> application does is parse arguments, initialize some context
> structures, load some authentication providers, and then ask
> libsvn_client to do the real work. What sort of things would
> libsvn_cmdline offer? I don't see the 'itch'.
>
> Chia-liang Kao wrote:
> > I guess it's mainly for those prompt callbacks used by the auth
> > provider functions.
>
> Those are awfully simple things -- something which merely prompts the
> user for a string? It seems a bit of overkill to create a whole
> shared library for such a thing, don't you think? I would imagine
> that writing something in python or perl would be *much* simpler than
> calling a C function.
It's easy to create a minimal Subversion client, but a full-fledged
client is more work. Sure, it's easy to add a few callbacks to prompt
the user for information, but when you add up all the little functions
you get a complex system. There's almost 9000 lines of code in the
clients/cmdline directory -- I don't think that's trivial at all.
Who would use libsvn_cmdline? Anyone who wants to add functionality to
the standard command-line client. If you want to just change a few
functions from the command-line client, it is very helpful to have
access to the rest of the implementation.
In order to allow the Python bindings to be tested by the standard
command-line test suite, I'm trying to rewrite the interesting
functionality from the Subversion command-line client in Python, piece
by piece. If I can implement and replace each function separately, it
will make my life a lot easier. This will only be possible if we have a
libsvn_cmdline library.
I'm sure the libsvn_cmdline library will help other people too. My
recommendation: Convert the current command-line client into a library
right now, but mark the API as experimental. Once people start using
the command-line functions as a library, we'll start to see what needs
to be abstracted.
Karl Fogel <kf...@collab.net> writes:
> David James <ja...@gmail.com> writes:
> > Questions:
> > - Is it OK to create the new libsvn_cmdline library? It will make
> > my command-line SWIG work this summer much easier.
> Personally, I would like to see some of the abstraction patches
> first. Those patches can include the new libsvn_cmdline/ subdir
> if necessary. Then, assuming the patches are okay to check in on the
> mainline, we can do so.
Works for me. I can send in my abstraction patches first. I'd like to
get the libsvn_cmdline library up and running soon, though. Even if we
just converted the current command-line client directly into a library,
without making any changes besides renaming the 'main' function, I
think this library would be quite useful to many people, including
myself.
Karl Fogel <kf...@collab.net> writes:
> > - Is it OK to use malloc / free in the SWIG bindings? In some
> > cases, this seems to be the only solution because APR pools are
> > not always available in a typemap.
> Isn't the solution to change the typemap? Are we using malloc/free in
> other bindings?
Here's an example of a function (from main.c) which is difficult to
Swigify without using malloc/free.
int svn_cmdline_main (int argc, const char * const *argv)
(This is the 'main' function from clients/cmdline/main.c)
When you call the svn_cmdline_main function using Python strings, we
need to allocate memory to store C-compatible versions of these
strings. We can easily accomplish this using malloc/free. Allocating a
brand new APR pool for this purpose would be very wasteful, especially
since the purpose of the svn_cmdline_main function already allocates
its own root pool. So, I think the best solution here is to use
malloc/free. What do you think?
svn_cmdline_main isn't the only function which suffers from this
problem. Here are some more examples from the bindings:
/* Convert Python strings to svn_string_buf_t * objects */
%typemap(python,in) svn_stringbuf_t * {
if (!PyString_Check($input)) {
PyErr_SetString(PyExc_TypeError, "not a string");
return NULL;
}
$1 = svn_stringbuf_ncreate(PyString_AS_STRING($input),
PyString_GET_SIZE($input),
/* ### gah... what pool to use? */
_global_pool);
}
The current typemap for svn_stringbuf_t uses the _global_pool variable,
even though this variable does not exist in all contexts. As a result,
our SWIG bindings are less complete than they could be. Right now,
our SWIG bindings ignore all functions and structures which use
string objects but do not use pools. Read below for details:
Re: Python Command-line Client Bindings
Posted by Ben Collins-Sussman <su...@collab.net>.
On Jun 29, 2005, at 8:39 AM, Chia-liang Kao wrote:
> Ben Collins-Sussman <sussman <at> collab.net> writes:
>
>> Take a look at minimal_client.c . All a commandline-based client
>> application does is parse arguments, initialize some context
>> structures, load some authentication providers, and then ask
>> libsvn_client to do the real work. What sort of things would
>> libsvn_cmdline offer? I don't see the 'itch'.
>>
>
> I guess it's mainly for those prompt callbacks used by the auth
> provider
> functions.
>
Those are awfully simple things -- something which merely prompts the
user for a string? It seems a bit of overkill to create a whole
shared library for such a thing, don't you think? I would imagine
that writing something in python or perl would be *much* simpler than
calling a C function.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by Chia-liang Kao <cl...@clkao.org>.
Ben Collins-Sussman <sussman <at> collab.net> writes:
> Take a look at minimal_client.c . All a commandline-based client
> application does is parse arguments, initialize some context
> structures, load some authentication providers, and then ask
> libsvn_client to do the real work. What sort of things would
> libsvn_cmdline offer? I don't see the 'itch'.
I guess it's mainly for those prompt callbacks used by the auth provider
functions.
Cheers,
CLK
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by Ben Collins-Sussman <su...@collab.net>.
On Jun 28, 2005, at 10:08 PM, kfogel@collab.net wrote:
>
> Are you positive you need libsvn_cmdline?
Yes, I'm a bit puzzled by this idea too, it feels like a bit of over-
abstraction to me.
Take a look at minimal_client.c . All a commandline-based client
application does is parse arguments, initialize some context
structures, load some authentication providers, and then ask
libsvn_client to do the real work. What sort of things would
libsvn_cmdline offer? I don't see the 'itch'.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by kf...@collab.net.
David James <ja...@gmail.com> writes:
> Questions:
> - Is it OK to create the new libsvn_cmdline library? It will make my
> command-line SWIG work this summer much easier.
Personally, I would like to see some of the abstraction patches first.
Those patches can include the new libsvn_cmdline/ subdir if necessary.
Then, assuming the patches are okay to check in on the mainline, we
can do so.
We (the project) need to do this in such a way that any changes
committed are still good even if you were to drop of the face of the
earth at any random moment. That's a general principle: even if
change X is checked in for the explicit purpose of facilitating change
Y down the road, change X still has to stand on its own merits,
because there's no guarantee that Y will ever happen. Y may be very
*likely* to happen -- as I think it is in this case -- but by
following the general principle, we can never go wrong :-).
Are you positive you need libsvn_cmdline? It seems plausible to me,
but maybe it would make more sense to start on the client, and then
see what you need as you go? However you want to do it is fine.
If we get to the point where no patch has been committed yet, but
you're having trouble managing your changes locally, I'm +1 on
creating a branch (or branches) for this.
> - Is it possible to mark the libsvn_cmdline_cl.h API as experimental
> and subject to change? I think that the API for the command-line
> client could go through quite a few changes and improvements this
> summer and I don't want to have to worry about revving the API yet.
I think that's fine -- see if there's an standard Doxygen way to do
it, though.
> - Is it OK to use malloc / free in the SWIG bindings? In some cases,
> this seems to be the only solution because APR pools are not always
> available in a typemap.
Isn't the solution to change the typemap? Are we using malloc/free in
other bindings?
> ----------------------------------------------------------------------
> Implementation Plan
> ----------------------------------------------------------------------
>
> 1. Write a simple script which creates a Python command-line parser
> based on the svn_cl__options and svn_cl__cmd_table structures from
> main.c in subversion/clients/cmdline. The Python standard optparse
> module will do most of the work, but we will also need to write
> custom code to parse Subversion revision numbers and ranges.
Does Subversion have these effectively abstracted already? Can that
just be librarized? Much, much better to avoid code duplication,
especially if other binding languages later want to do the same thing
you're doing for Python.
> 2. Upgrade our script to dispatch each command to the appropriate
> command-line client function. This initial prototype will provide
> the full functionality of the Subversion client, but will only test
> the surface functionality of SWIG.
>
> 3. Upgrade the Subversion automated test suite to test our new script
> using the command-line test suite. Fix any errors that are found.
>
> 4. Replace each command-line C function in the Python command-line
> client with an appropriate Python implementation. Each function
> should be implemented, tested, and committed as a separate patch. If
> adding a new function reveals a bug in the underlying SWIG/Python
> library, these bugs should be reported to the Subversion development
> list.
Okay, I see how this code duplication (item 4) is necessary to achieve
the goal of getting the public APIs tested.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Python Command-line Client Bindings
Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2005-06-28 at 22:33 -0400, David James wrote:
> * Create a libsvn_cmdline library
Along with other developers, I'm not happy with this idea. I think it
would just wind up being a libsvn_client with a different style of API.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org