You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by Nick Wellnhofer <we...@aevum.de> on 2014/08/26 12:15:46 UTC

[lucy-dev] C API documentation in POD format

Lucifers,

I'd like to add the Clownfish C API documentation to our generated Perl PODs, 
so developers of Perl extensions can find all the docs in one place. I'm 
thinking of creating POD files like Clownfish/Docs/C/VArray.pod for every 
Clownfish class. There are some classes like LockFreeRegistry that shouldn't 
be exposed, maybe by marking them private. But we can deal with that later.

For Lucy extension writers, it would also be useful to have the C API 
documented in POD format. But I don't want to clutter the documentation with 
another 150 files. Maybe we could organize the Lucy C API docs by 
subdirectory, like Lucy/Docs/C/Search.pod.

Nick

Re: [lucy-dev] C API documentation in POD format

Posted by Philip Southam <ph...@zefr.com>.
As a Lucy extension writer for Go, I would love to see docs for the C API.
Currently I've got to grope around the header files for both clownfish and
Lucy in order to figure out functionality, which is not as fun as it sounds.

Philip Southam
Chief Architect / Яeverse Эngineer
http://zefr.com
On Aug 26, 2014 3:16 AM, "Nick Wellnhofer" <we...@aevum.de> wrote:

> Lucifers,
>
> I'd like to add the Clownfish C API documentation to our generated Perl
> PODs, so developers of Perl extensions can find all the docs in one place.
> I'm thinking of creating POD files like Clownfish/Docs/C/VArray.pod for
> every Clownfish class. There are some classes like LockFreeRegistry that
> shouldn't be exposed, maybe by marking them private. But we can deal with
> that later.
>
> For Lucy extension writers, it would also be useful to have the C API
> documented in POD format. But I don't want to clutter the documentation
> with another 150 files. Maybe we could organize the Lucy C API docs by
> subdirectory, like Lucy/Docs/C/Search.pod.
>
> Nick
>

Re: [lucy-dev] C API documentation in POD format

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Wed, Aug 27, 2014 at 3:57 AM, Nick Wellnhofer <we...@aevum.de> wrote:
> Can't we just publish the Clownfish API with a disclaimer that it's subject
> to change, and keep working on it at everyone's preferred pace?

+1

I'm making you work too hard by protesting too much.  Let's go with your
plan -- I'm cool with it.  I agree with both you and Peter that Lucy API
publication has had its issues.

> But the last thing I want is to pressure you into
> putting anything aside just to finish other language bindings as soon as
> possible. This doesn't sound like fun, and that's what it all should be
> about.

Marvin's problems are nice problems to have:  Which super fun and rewarding
activity do I not have time for this month?

Marvin Humphrey

Re: [lucy-dev] C API documentation in POD format

Posted by Peter Karman <pe...@peknet.com>.
Nick Wellnhofer wrote on 8/27/14 5:57 AM:

> Can't we just publish the Clownfish API with a disclaimer that it's subject to
> change, and keep working on it at everyone's preferred pace? I don't see a
> problem with that. But the last thing I want is to pressure you into putting
> anything aside just to finish other language bindings as soon as possible. This
> doesn't sound like fun, and that's what it all should be about.

+1

Nick's points about the target audience for Clownfish and the rate of adoption
are correct, IMO. Let's not make the perfect the enemy of the good. Nothing
helps clarify the warts in an API like documenting it and getting people to use
it based on the documentation.


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: [lucy-dev] C API documentation in POD format

Posted by Nick Wellnhofer <we...@aevum.de>.
On 27/08/2014 09:20, Marvin Humphrey wrote:
> On Tue, Aug 26, 2014 at 2:23 PM, Nick Wellnhofer <we...@aevum.de> wrote:
>
>> What’s so bad about releasing new versions that break compatibility?
>
> I'm cool with it, I just think it's unrealistic.  Once people start using the
> software, the social costs of breaking compat make it increasingly difficult
> to change things.
>
> Superficial changes which are nonetheless justified because they make APIs
> more consistent or simpler don't go over well with people who just want stuff
> to work and have to cope with disruption.  Delaying API publication until as
> many known mistakes are ironed out as possible means less conflict and yields
> a better product.

I would expect only very few early adopters of Clownfish in the first year, if 
we get any at all. Also, the decision to use Clownfish in a new project or to 
even make the switch with an existing project will usually take some time. By 
then, we might have a more mature release out anyway. My guess is that there 
won't be serious users in first six months. We'd simply get a considerable 
head start if we opened up the Clownfish internals now.

Besides, Clownfish targets skilled developers, not the crowd who just wants 
stuff to work. These users should be able to deal with API changes. And even 
if we have to make disruptive changes, I'd rather have users who are unhappy 
with some decisions than no users at all.

Then we have developers like Philip who work with Clownfish right now and must 
wade through the .cfh files and other parts of the source code to find the 
documentation they need. Do we really want to make life harder for them just 
to please some hypothetical future users?

Another feature that is useless without proper Clownfish documentation is the 
support for C-based Lucy extensions introduced in 0.4.

Can't we just publish the Clownfish API with a disclaimer that it's subject to 
change, and keep working on it at everyone's preferred pace? I don't see a 
problem with that. But the last thing I want is to pressure you into putting 
anything aside just to finish other language bindings as soon as possible. 
This doesn't sound like fun, and that's what it all should be about.

Nick


Re: [lucy-dev] C API documentation in POD format

Posted by Peter Karman <pe...@peknet.com>.
Marvin Humphrey wrote on 8/27/14 2:20 AM:
> On Tue, Aug 26, 2014 at 2:23 PM, Nick Wellnhofer <we...@aevum.de> wrote:

>> What’s so bad about releasing new versions that break compatibility?
> 
> I'm cool with it, I just think it's unrealistic.  Once people start using the
> software, the social costs of breaking compat make it increasingly difficult
> to change things.
> 

Difficult but not impossible, and when determining the social calculus, we must
take into account that early adopters expect to bleed. It's one of the
trade-offs for getting cool stuff early. I jumped into KS-then-Lucy because I
recognized it as a space I wanted to play in, and I knew I would have to be
flexible in my adoption of it because that's how OSS works.

I believe that one of the things that has hampered Lucy's growth is that we've
been too conservative about releasing things and encouraging people to play with
them. Docs encourage play.


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: [lucy-dev] C API documentation in POD format

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, Aug 26, 2014 at 2:23 PM, Nick Wellnhofer <we...@aevum.de> wrote:

> Do you really want to complete the development of Python and Ruby bindings
> first before telling people that they can start using Clownfish?

Well yeah -- but if I haven't been persuasive enough, I'd rather go along with
your plan.

> At the current pace of development, this might take a couple of years.

If I could work uninterrupted, Python and Ruby would probably take me a month
each from where they are now.

I guess that just means I have to work as hard as possible now to get stuff in
before the concrete stiffens.  And I have to be lucky about avoiding
distractions.

> What’s so bad about releasing new versions that break compatibility?

I'm cool with it, I just think it's unrealistic.  Once people start using the
software, the social costs of breaking compat make it increasingly difficult
to change things.

Superficial changes which are nonetheless justified because they make APIs
more consistent or simpler don't go over well with people who just want stuff
to work and have to cope with disruption.  Delaying API publication until as
many known mistakes are ironed out as possible means less conflict and yields
a better product.

Marvin Humphrey

Re: [lucy-dev] C API documentation in POD format

Posted by Nick Wellnhofer <we...@aevum.de>.
On Aug 26, 2014, at 19:08 , Marvin Humphrey <ma...@rectangular.com> wrote:

> On Tue, Aug 26, 2014 at 3:15 AM, Nick Wellnhofer <we...@aevum.de> wrote:
> 
>> There are some classes like LockFreeRegistry that shouldn't be exposed,
>> maybe by marking them private. But we can deal with that later.
> 
> Hmm.  I don't understand this approach to publishing an API.

What I meant is that we can first write the code that generates the docs, then think about what parts of the API we want to make public.

> Clownfish is likely to remain in flux for a while yet -- but if we expose a
> public API, we're making backwards compatibility promises which either
> constrain our ability to evolve the library, or if we break them, try the
> patience of our users.
> 
> Forestalling the publication of an immature API and finishing Clownfish's
> fundamental design is important enough to me that I've put other projects on
> hold (including the Apache release policy clarification initiative and the
> Lucy Book Club).  Disappointing my collaborators in those other projects was
> painful, but it was the right move -- because you've invested so much, Nick,
> and because I believe that the Clownfish "symbiotic" approach is important in
> this era of siloed language ecosystems.
> 
> So please help me to understand where you want to go.  Do you want to freeze
> the API?  Do you want to simply break compat later?
> 
> My priority would be to write Clownfish bindings for Python and Ruby, and
> optionally Go.  Once we finish those, I think we will have understood enough
> of the challenges of writing a symbiotic object system that we can make nearly
> optimal engineering compromises.

My plan is to publish an immature API and break compatibility when we release new versions of Clownfish. In my opinion, Clownfish 0.4.0 is a usable product. With Lucy, we have a mature and complex downstream project after all. I primarily want to attract new developers and I don’t think this will work if we keep Clownfish as a black-box implementation detail of Lucy. Do you really want to complete the development of Python and Ruby bindings first before telling people that they can start using Clownfish? At the current pace of development, this might take a couple of years.

What’s so bad about releasing new versions that break compatibility? Potential users should be made aware that Clownfish is in flux, but they can continue to use any old version they like. I don’t think maintaining old releases will be much work, so what do we have to lose? We could gain much more from any bit of additional visibility.

Nick


Re: [lucy-dev] C API documentation in POD format

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, Aug 26, 2014 at 3:15 AM, Nick Wellnhofer <we...@aevum.de> wrote:

> I'd like to add the Clownfish C API documentation to our generated Perl
> PODs, so developers of Perl extensions can find all the docs in one place.
> I'm thinking of creating POD files like Clownfish/Docs/C/VArray.pod for
> every Clownfish class.

Once the API is mature, this is a great plan!

> There are some classes like LockFreeRegistry that shouldn't be exposed,
> maybe by marking them private. But we can deal with that later.

Hmm.  I don't understand this approach to publishing an API.

Clownfish is likely to remain in flux for a while yet -- but if we expose a
public API, we're making backwards compatibility promises which either
constrain our ability to evolve the library, or if we break them, try the
patience of our users.

Forestalling the publication of an immature API and finishing Clownfish's
fundamental design is important enough to me that I've put other projects on
hold (including the Apache release policy clarification initiative and the
Lucy Book Club).  Disappointing my collaborators in those other projects was
painful, but it was the right move -- because you've invested so much, Nick,
and because I believe that the Clownfish "symbiotic" approach is important in
this era of siloed language ecosystems.

So please help me to understand where you want to go.  Do you want to freeze
the API?  Do you want to simply break compat later?

My priority would be to write Clownfish bindings for Python and Ruby, and
optionally Go.  Once we finish those, I think we will have understood enough
of the challenges of writing a symbiotic object system that we can make nearly
optimal engineering compromises.

Marvin Humphrey