You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Robert Newson <rn...@apache.org> on 2012/11/01 00:40:40 UTC
Re: CouchDB Plugins First Draft
I quite like the rabbitmq approach (a lot like the apache httpd approach).
you can enable/disable/list plugins but the tool doesn't include the
downloading and managed repository bits, which is a huge rabbit hole (pun
intended).
On 31 October 2012 22:28, Alexander Shorin <kx...@gmail.com> wrote:
> Hi Jan.
>
> More detailed and explained question from IRC meeting.
>
> Why there is need to reinvent own package manager when most modern
> systems (even Windows) already has package manager?
>
>
> What problem tried to be solved?
> ------------------------------------------------------------
>
> Easy find, install and activate any CouchDB plugin in one-shot action,
> right? Following example from gist:
>
> $ cpm # couch plugin manager
> $ cpm search geocouch
> $ cpm install geocouch
>
> While it tries to simplify extending of CouchDB, it brings a lot of
> problems into system
>
>
> What problems cpm brings?
> -----------------------------------------------------
>
> 1. Dependencies.
>
> While any system package already able to resolve dependencies,
> conflicts, provide slots for different versions, cpm will need to
> reimplement this features. Geocouch isn't good example, next one is
> much better:
>
> $ cpm install couchdb-lucene # what about dependencies?
>
> I don't think (and hope it will not) that this command will install
> JRE and other java things into my system while I'm only expects this
> plugin. System package managers already know what need and which
> versions of to properly install lucene. Should this `cpm install`
> command only check for suitable environment before landing? Or may be
> cpm will know how to cooperate with system package managers to propose
> install required dependencies?
>
> 2. Namespace.
>
> Since CouchDB installs system-wide (most cases) should plugins be
> installed in same way? Or each user may have his own set of active
> plugins? But it looks like that only CouchDB admin may operate with
> plugins from Futon, so plugins have to live at least by path
> accessiable for couchdb user. So there couldn't be personalised
> plugins and npm system wouldn't work. So, cpm utility is only useful
> for root user which is already has access to system package manager.
>
> 3. BigCouch
>
> How plugins system proposed to be work after BigCouch merge? Would
> plugins be installed for all shards or only just for current one? Will
> cluster works right if some shards would have geocouch-1.0, some of
> them has geocouch-1.3 with breaking changes and for others it just
> missed?
>
> Why system package manager?
> ---------------------------------------------------------
>
> $ flaggie dev-db/couchdb +lucene # or echo "dev-db/couchdb lucene" >>
> /etc/portage/package.use
> $ emerge couchdb
>
> And we'd got CouchDB with couchdb-lucene plugin and handled all
> required dependencies. Now system takes care about packages
> consistency and we're always sure and accidental iceadtea uninstall
> would be passed silently since it have to break our couchdb-lucene
> extension.
>
> In same way other projects are already have implemented their
> "plugins" system: nginx, php. Yes, they have not plugins, but modules,
> but since they acts on system level there is no easy and secure way to
> let everyone enable/disable/install any thirdparty component and this
> way is the right one.
>
> Following RabbitMQ project, they ships with all main plugins on board
> - you need just to enable with through cli utility. If you need to
> install some third party plugin, just copy his directory to rabbits
> plugins one. But to be honest, RabbitMQ plugin manager tool doesn't
> tries to act as package manager: it only shows, enables and disables
> plugins, nothing more - that's why their approach works fine.
>
>
> So I've smoothly went from original question to set of other ones, but
> I suppose they are also interested to be answered since they are
> defined show cpm will work and handle plugins.
>
> P.S. Sorry for poorly mind dump style, 2:40 am there. I could try to
> explain something in other way if it wasn't clear enough...
>
> --
> ,,,^..^,,,
>
>
> On Mon, Oct 29, 2012 at 8:41 PM, Jan Lehnardt <ja...@apache.org> wrote:
> > Hey all,
> >
> > during one of the past IRC Meetings I promised to write up
> > what I have in mind for a CouchDB Plugin System.
> >
> > Here is my first draft:
> >
> > https://gist.github.com/3974676
> >
> > This is very early stage, but I wanted to get the idea out
> > of the door. If anyone feels compelled to go into further
> > detail, or even implement some of this, more power to you!
> >
> > Note: This is especially interesting for the non-Erlang
> > folks here that need an excuse to contribute :)
> >
> > For the frontend, we’d need some mockups on how the Futon
> > interaction would work (that is regardless of the state
> > of Futon.Current and Futon.Next).
> >
> > On the backend, you could start figuring out what the
> > registry could look like.
> >
> > I’m looking forward to your feedback!
> >
> > Cheers!
> > Jan
> > --
> >
>
Re: CouchDB Plugins First Draft
Posted by Robert Newson <rn...@apache.org>.
My only thought is how well (if at all) it fits into an OTP shape.
On 31 October 2012 23:40, Robert Newson <rn...@apache.org> wrote:
> I quite like the rabbitmq approach (a lot like the apache httpd approach).
> you can enable/disable/list plugins but the tool doesn't include the
> downloading and managed repository bits, which is a huge rabbit hole (pun
> intended).
>
>
>
> On 31 October 2012 22:28, Alexander Shorin <kx...@gmail.com> wrote:
>
>> Hi Jan.
>>
>> More detailed and explained question from IRC meeting.
>>
>> Why there is need to reinvent own package manager when most modern
>> systems (even Windows) already has package manager?
>>
>>
>> What problem tried to be solved?
>> ------------------------------------------------------------
>>
>> Easy find, install and activate any CouchDB plugin in one-shot action,
>> right? Following example from gist:
>>
>> $ cpm # couch plugin manager
>> $ cpm search geocouch
>> $ cpm install geocouch
>>
>> While it tries to simplify extending of CouchDB, it brings a lot of
>> problems into system
>>
>>
>> What problems cpm brings?
>> -----------------------------------------------------
>>
>> 1. Dependencies.
>>
>> While any system package already able to resolve dependencies,
>> conflicts, provide slots for different versions, cpm will need to
>> reimplement this features. Geocouch isn't good example, next one is
>> much better:
>>
>> $ cpm install couchdb-lucene # what about dependencies?
>>
>> I don't think (and hope it will not) that this command will install
>> JRE and other java things into my system while I'm only expects this
>> plugin. System package managers already know what need and which
>> versions of to properly install lucene. Should this `cpm install`
>> command only check for suitable environment before landing? Or may be
>> cpm will know how to cooperate with system package managers to propose
>> install required dependencies?
>>
>> 2. Namespace.
>>
>> Since CouchDB installs system-wide (most cases) should plugins be
>> installed in same way? Or each user may have his own set of active
>> plugins? But it looks like that only CouchDB admin may operate with
>> plugins from Futon, so plugins have to live at least by path
>> accessiable for couchdb user. So there couldn't be personalised
>> plugins and npm system wouldn't work. So, cpm utility is only useful
>> for root user which is already has access to system package manager.
>>
>> 3. BigCouch
>>
>> How plugins system proposed to be work after BigCouch merge? Would
>> plugins be installed for all shards or only just for current one? Will
>> cluster works right if some shards would have geocouch-1.0, some of
>> them has geocouch-1.3 with breaking changes and for others it just
>> missed?
>>
>> Why system package manager?
>> ---------------------------------------------------------
>>
>> $ flaggie dev-db/couchdb +lucene # or echo "dev-db/couchdb lucene" >>
>> /etc/portage/package.use
>> $ emerge couchdb
>>
>> And we'd got CouchDB with couchdb-lucene plugin and handled all
>> required dependencies. Now system takes care about packages
>> consistency and we're always sure and accidental iceadtea uninstall
>> would be passed silently since it have to break our couchdb-lucene
>> extension.
>>
>> In same way other projects are already have implemented their
>> "plugins" system: nginx, php. Yes, they have not plugins, but modules,
>> but since they acts on system level there is no easy and secure way to
>> let everyone enable/disable/install any thirdparty component and this
>> way is the right one.
>>
>> Following RabbitMQ project, they ships with all main plugins on board
>> - you need just to enable with through cli utility. If you need to
>> install some third party plugin, just copy his directory to rabbits
>> plugins one. But to be honest, RabbitMQ plugin manager tool doesn't
>> tries to act as package manager: it only shows, enables and disables
>> plugins, nothing more - that's why their approach works fine.
>>
>>
>> So I've smoothly went from original question to set of other ones, but
>> I suppose they are also interested to be answered since they are
>> defined show cpm will work and handle plugins.
>>
>> P.S. Sorry for poorly mind dump style, 2:40 am there. I could try to
>> explain something in other way if it wasn't clear enough...
>>
>> --
>> ,,,^..^,,,
>>
>>
>> On Mon, Oct 29, 2012 at 8:41 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> > Hey all,
>> >
>> > during one of the past IRC Meetings I promised to write up
>> > what I have in mind for a CouchDB Plugin System.
>> >
>> > Here is my first draft:
>> >
>> > https://gist.github.com/3974676
>> >
>> > This is very early stage, but I wanted to get the idea out
>> > of the door. If anyone feels compelled to go into further
>> > detail, or even implement some of this, more power to you!
>> >
>> > Note: This is especially interesting for the non-Erlang
>> > folks here that need an excuse to contribute :)
>> >
>> > For the frontend, we’d need some mockups on how the Futon
>> > interaction would work (that is regardless of the state
>> > of Futon.Current and Futon.Next).
>> >
>> > On the backend, you could start figuring out what the
>> > registry could look like.
>> >
>> > I’m looking forward to your feedback!
>> >
>> > Cheers!
>> > Jan
>> > --
>> >
>>
>
>
Re: CouchDB Plugins First Draft
Posted by Benoit Chesneau <bc...@gmail.com>.
well do we want a plugin system or just a way to load apps at startup in
that case just point ERL_FLAGS in default.ini in your app and done. Which
is what does rabbitmq basically.
- benoît
On Thu, Nov 1, 2012 at 10:30 AM, Simon Metson <si...@cloudant.com> wrote:
> +1 - keep it simple for the first iteration.
>
>
> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>
> > I quite like the rabbitmq approach (a lot like the apache httpd
> approach).
> > you can enable/disable/list plugins but the tool doesn't include the
> > downloading and managed repository bits, which is a huge rabbit hole (pun
> > intended).
>
>
>
>
Re: CouchDB Plugins First Draft
Posted by Jan Lehnardt <ja...@apache.org>.
On Nov 1, 2012, at 16:53 , Bob Dionne <di...@dionne-associates.com> wrote:
>
> On Nov 1, 2012, at 7:53 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com> wrote:
>>
>>> Reminds me of my favorite book - "Sketches of an Elephant"
>>>
>>> Jan, thanks for putting a stake in the ground, I've wanted to see this forever. The proposal in my mind takes too much of a product management or marketing view (perhaps knowingly). Here's how it will look the buttons one will push, etc..
>>
>> Totally knowingly, intentional even.
>
> in that case, good luck, that's a long expensive haul.
>
>>
>>> I think the "what" and "how it works" are important to decide on first, .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs for that, the "what" is much clearer.
>>
>> This seems contradictory to your previous statement
>
>> My document started the "what" and "how it works" discussion just fine. Whatever is unclear needs to be resolved *before* we jump into any implementation.
>>
>>
>>> Looking over the efforts to date, couchdb-lucene, and geocouch, these two are quite different in terms of design, one is roughly loosely coupled, the other more native (in the same VM).
>>
>> Yup, we need to define how this fits into the plugin system. Maybe we never to something like couchdb-lucene, maybe we do native plugins first, and external plugins later, or the other way around. Thanks for making this more explicit, I will add this to the document.
>>
>>
>>> A plugin architecture, in my mind, should emerge from the code refactoring and layout we're currently discussing.
>>
>> I respectfully disagree. I would like to start from the user and work my way down. What ever internal refactorings make sense to support the use-case, we will have to make. I trust that we are smart enough to make this in a way that is favourable to the rest of the code base.
>
> I see, great. I think we have perhaps different interpretations of "user". Given the current state of the code base I see the users as programmers trying to extend the existing code in interesting ways. An architecture for plugins that emerges bottoms up from those attempts, similar to how the couch_api_wrap and couch_index refactoring came about, is what I'm interested in. Top down high level approaches rarely work in practice *except* where there are lots of resources and control over the process, both of which are in short supply in open source efforts.
Yup, different POV, my users are people who program against CouchDB that might want additional features that aren’t in core CouchDB. There are also plugin authors to which we need to cater, and the things you bring up definitely fall in that bucket. I just want to start things out from the end-user, that is all. We are on the same page.
Cheers
Jan
--
Re: CouchDB Plugins First Draft
Posted by Bob Dionne <di...@dionne-associates.com>.
On Nov 1, 2012, at 7:53 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com> wrote:
>
>> Reminds me of my favorite book - "Sketches of an Elephant"
>>
>> Jan, thanks for putting a stake in the ground, I've wanted to see this forever. The proposal in my mind takes too much of a product management or marketing view (perhaps knowingly). Here's how it will look the buttons one will push, etc..
>
> Totally knowingly, intentional even.
in that case, good luck, that's a long expensive haul.
>
>> I think the "what" and "how it works" are important to decide on first, .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs for that, the "what" is much clearer.
>
> This seems contradictory to your previous statement
> My document started the "what" and "how it works" discussion just fine. Whatever is unclear needs to be resolved *before* we jump into any implementation.
>
>
>> Looking over the efforts to date, couchdb-lucene, and geocouch, these two are quite different in terms of design, one is roughly loosely coupled, the other more native (in the same VM).
>
> Yup, we need to define how this fits into the plugin system. Maybe we never to something like couchdb-lucene, maybe we do native plugins first, and external plugins later, or the other way around. Thanks for making this more explicit, I will add this to the document.
>
>
>> A plugin architecture, in my mind, should emerge from the code refactoring and layout we're currently discussing.
>
> I respectfully disagree. I would like to start from the user and work my way down. What ever internal refactorings make sense to support the use-case, we will have to make. I trust that we are smart enough to make this in a way that is favourable to the rest of the code base.
I see, great. I think we have perhaps different interpretations of "user". Given the current state of the code base I see the users as programmers trying to extend the existing code in interesting ways. An architecture for plugins that emerges bottoms up from those attempts, similar to how the couch_api_wrap and couch_index refactoring came about, is what I'm interested in. Top down high level approaches rarely work in practice *except* where there are lots of resources and control over the process, both of which are in short supply in open source efforts.
YMMV,
Bob
>
> I definitely want to stress that I’d like to define the plugin feature from the user down, not from the technology up. This doesn’t mean we’ll bend over backwards to accommodate some crazy concept we draw up, but I think it helps keeping in mind who we do this for is great for making informed decisions rather than what’s easier for the code we have toda
>
>
>> We should ask and answer questions such as:
>>
>> 1. the role of externals
>>
>> 2 access to the storage layer (API?)
>>
>> 3. separation from http layer
>>
>> 4. support for code upgrades
>>
>> 5. balancing of resources
>>
>> I didn't mention Auth and Logging as I think these are separate in terms of concerns, more system oriented. Whereas geocouch and couchdb-lucene are actually extending the functionality in meaningful ways.
>
> Excellent observations and points!
>
> Cheers
> Jan
> --
>
>>
>>
>> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
>>
>>> +1 - keep it simple for the first iteration.
>>>
>>>
>>> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>>>
>>>> I quite like the rabbitmq approach (a lot like the apache httpd approach).
>>>> you can enable/disable/list plugins but the tool doesn't include the
>>>> downloading and managed repository bits, which is a huge rabbit hole (pun
>>>> intended).
>>>
>>>
>>>
>>
>
Re: CouchDB Plugins First Draft
Posted by Noah Slater <ns...@apache.org>.
This is a great initiative. Thanks for leading the charge, Jan!
I am personally very excited about the future of CouchDB with a lean core
and a vibrant plugin system.
On 1 November 2012 12:08, Jan Lehnardt <ja...@apache.org> wrote:
>
> On Nov 1, 2012, at 12:57 , Robert Newson <rn...@apache.org> wrote:
>
> > couchdb-lucene already "plugs in" to couchdb in a pretty reasonable way,
> so
> > I don't think it illuminates this discussion. It will always require an
> > external process (the JVM). It's hard to see how a plugin system could be
> > so generic as to support every possible kind of plugin. I quite like the
> > rabbitmq one that insists that each plugin is OTP-shaped, and stopping at
> > that point.
>
> Cool, thanks. I think a “native plugins only” first step would be a great
> thing to ship. Discussion of “external plugins” can happen after that. I’ll
> make a note about priorities in my document.
>
>
> > Apart from that observation, I don't have much to add. I've not
> > suffered from the lack of "plugin support" and will be no richer for it
> > when it lands.
>
> You’ll be surprised what cool things are going to happen :)
>
> My motivation for this is clearing out all these branches all of us
> have with features that are quite nice, but not really applicable to
> everybody, or not feasible to carry around in core.
>
> I also hope that this leads to more experimental features on top
> of CouchDB that solve a number of people’s problem. Things like
> a transactional _bulk_docs plugin for people the want that and
> understand the trade-offs (look at me opening a particularly
> big can of worms).
>
> I’d like a future where we have a solution for developer’s problems
> rather than telling them to rethink their problem-space like we do
> now. A plugin system allows us to do that without compromising the
> integrity of core Apache CouchDB, while making our technology more
> applicable to a wider range of users.
>
> Finally, I think that we can use this mechanism to move some
> features out of what is currently core (discussion TBD for when we
> *do have* plugins, please don’t jump onto that particular point now,
> thanks :) and make for a leaner code base all around.
>
> * * *
>
> I’m happy to concede that you don’t agree with this future, I’m just
> sharing my unbounded excitement for this :)
>
> Cheers
> Jan
> --
>
>
>
>
>
> >
> > On 1 November 2012 11:53, Jan Lehnardt <ja...@apache.org> wrote:
> >
> >>
> >> On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com>
> >> wrote:
> >>
> >>> Reminds me of my favorite book - "Sketches of an Elephant"
> >>>
> >>> Jan, thanks for putting a stake in the ground, I've wanted to see this
> >> forever. The proposal in my mind takes too much of a product management
> or
> >> marketing view (perhaps knowingly). Here's how it will look the buttons
> one
> >> will push, etc..
> >>
> >> Totally knowingly, intentional even.
> >>
> >>> I think the "what" and "how it works" are important to decide on first,
> >> .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs
> >> for that, the "what" is much clearer.
> >>
> >> This seems contradictory to your previous statement. My document started
> >> the "what" and "how it works" discussion just fine. Whatever is unclear
> >> needs to be resolved *before* we jump into any implementation.
> >>
> >>
> >>> Looking over the efforts to date, couchdb-lucene, and geocouch, these
> >> two are quite different in terms of design, one is roughly loosely
> coupled,
> >> the other more native (in the same VM).
> >>
> >> Yup, we need to define how this fits into the plugin system. Maybe we
> >> never to something like couchdb-lucene, maybe we do native plugins
> first,
> >> and external plugins later, or the other way around. Thanks for making
> this
> >> more explicit, I will add this to the document.
> >>
> >>
> >>> A plugin architecture, in my mind, should emerge from the code
> >> refactoring and layout we're currently discussing.
> >>
> >> I respectfully disagree. I would like to start from the user and work my
> >> way down. What ever internal refactorings make sense to support the
> >> use-case, we will have to make. I trust that we are smart enough to make
> >> this in a way that is favourable to the rest of the code base.
> >>
> >> I definitely want to stress that I’d like to define the plugin feature
> >> from the user down, not from the technology up. This doesn’t mean we’ll
> >> bend over backwards to accommodate some crazy concept we draw up, but I
> >> think it helps keeping in mind who we do this for is great for making
> >> informed decisions rather than what’s easier for the code we have today.
> >>
> >>
> >>> We should ask and answer questions such as:
> >>>
> >>> 1. the role of externals
> >>>
> >>> 2 access to the storage layer (API?)
> >>>
> >>> 3. separation from http layer
> >>>
> >>> 4. support for code upgrades
> >>>
> >>> 5. balancing of resources
> >>>
> >>> I didn't mention Auth and Logging as I think these are separate in
> terms
> >> of concerns, more system oriented. Whereas geocouch and couchdb-lucene
> are
> >> actually extending the functionality in meaningful ways.
> >>
> >> Excellent observations and points!
> >>
> >> Cheers
> >> Jan
> >> --
> >>
> >>>
> >>>
> >>> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
> >>>
> >>>> +1 - keep it simple for the first iteration.
> >>>>
> >>>>
> >>>> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
> >>>>
> >>>>> I quite like the rabbitmq approach (a lot like the apache httpd
> >> approach).
> >>>>> you can enable/disable/list plugins but the tool doesn't include the
> >>>>> downloading and managed repository bits, which is a huge rabbit hole
> >> (pun
> >>>>> intended).
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
>
--
NS
Re: CouchDB Plugins First Draft
Posted by Jan Lehnardt <ja...@apache.org>.
On Nov 1, 2012, at 12:57 , Robert Newson <rn...@apache.org> wrote:
> couchdb-lucene already "plugs in" to couchdb in a pretty reasonable way, so
> I don't think it illuminates this discussion. It will always require an
> external process (the JVM). It's hard to see how a plugin system could be
> so generic as to support every possible kind of plugin. I quite like the
> rabbitmq one that insists that each plugin is OTP-shaped, and stopping at
> that point.
Cool, thanks. I think a “native plugins only” first step would be a great
thing to ship. Discussion of “external plugins” can happen after that. I’ll
make a note about priorities in my document.
> Apart from that observation, I don't have much to add. I've not
> suffered from the lack of "plugin support" and will be no richer for it
> when it lands.
You’ll be surprised what cool things are going to happen :)
My motivation for this is clearing out all these branches all of us
have with features that are quite nice, but not really applicable to
everybody, or not feasible to carry around in core.
I also hope that this leads to more experimental features on top
of CouchDB that solve a number of people’s problem. Things like
a transactional _bulk_docs plugin for people the want that and
understand the trade-offs (look at me opening a particularly
big can of worms).
I’d like a future where we have a solution for developer’s problems
rather than telling them to rethink their problem-space like we do
now. A plugin system allows us to do that without compromising the
integrity of core Apache CouchDB, while making our technology more
applicable to a wider range of users.
Finally, I think that we can use this mechanism to move some
features out of what is currently core (discussion TBD for when we
*do have* plugins, please don’t jump onto that particular point now,
thanks :) and make for a leaner code base all around.
* * *
I’m happy to concede that you don’t agree with this future, I’m just
sharing my unbounded excitement for this :)
Cheers
Jan
--
>
> On 1 November 2012 11:53, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com>
>> wrote:
>>
>>> Reminds me of my favorite book - "Sketches of an Elephant"
>>>
>>> Jan, thanks for putting a stake in the ground, I've wanted to see this
>> forever. The proposal in my mind takes too much of a product management or
>> marketing view (perhaps knowingly). Here's how it will look the buttons one
>> will push, etc..
>>
>> Totally knowingly, intentional even.
>>
>>> I think the "what" and "how it works" are important to decide on first,
>> .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs
>> for that, the "what" is much clearer.
>>
>> This seems contradictory to your previous statement. My document started
>> the "what" and "how it works" discussion just fine. Whatever is unclear
>> needs to be resolved *before* we jump into any implementation.
>>
>>
>>> Looking over the efforts to date, couchdb-lucene, and geocouch, these
>> two are quite different in terms of design, one is roughly loosely coupled,
>> the other more native (in the same VM).
>>
>> Yup, we need to define how this fits into the plugin system. Maybe we
>> never to something like couchdb-lucene, maybe we do native plugins first,
>> and external plugins later, or the other way around. Thanks for making this
>> more explicit, I will add this to the document.
>>
>>
>>> A plugin architecture, in my mind, should emerge from the code
>> refactoring and layout we're currently discussing.
>>
>> I respectfully disagree. I would like to start from the user and work my
>> way down. What ever internal refactorings make sense to support the
>> use-case, we will have to make. I trust that we are smart enough to make
>> this in a way that is favourable to the rest of the code base.
>>
>> I definitely want to stress that I’d like to define the plugin feature
>> from the user down, not from the technology up. This doesn’t mean we’ll
>> bend over backwards to accommodate some crazy concept we draw up, but I
>> think it helps keeping in mind who we do this for is great for making
>> informed decisions rather than what’s easier for the code we have today.
>>
>>
>>> We should ask and answer questions such as:
>>>
>>> 1. the role of externals
>>>
>>> 2 access to the storage layer (API?)
>>>
>>> 3. separation from http layer
>>>
>>> 4. support for code upgrades
>>>
>>> 5. balancing of resources
>>>
>>> I didn't mention Auth and Logging as I think these are separate in terms
>> of concerns, more system oriented. Whereas geocouch and couchdb-lucene are
>> actually extending the functionality in meaningful ways.
>>
>> Excellent observations and points!
>>
>> Cheers
>> Jan
>> --
>>
>>>
>>>
>>> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
>>>
>>>> +1 - keep it simple for the first iteration.
>>>>
>>>>
>>>> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>>>>
>>>>> I quite like the rabbitmq approach (a lot like the apache httpd
>> approach).
>>>>> you can enable/disable/list plugins but the tool doesn't include the
>>>>> downloading and managed repository bits, which is a huge rabbit hole
>> (pun
>>>>> intended).
>>>>
>>>>
>>>>
>>>
>>
>>
Re: CouchDB Plugins First Draft
Posted by Robert Newson <rn...@apache.org>.
couchdb-lucene already "plugs in" to couchdb in a pretty reasonable way, so
I don't think it illuminates this discussion. It will always require an
external process (the JVM). It's hard to see how a plugin system could be
so generic as to support every possible kind of plugin. I quite like the
rabbitmq one that insists that each plugin is OTP-shaped, and stopping at
that point. Apart from that observation, I don't have much to add. I've not
suffered from the lack of "plugin support" and will be no richer for it
when it lands.
On 1 November 2012 11:53, Jan Lehnardt <ja...@apache.org> wrote:
>
> On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com>
> wrote:
>
> > Reminds me of my favorite book - "Sketches of an Elephant"
> >
> > Jan, thanks for putting a stake in the ground, I've wanted to see this
> forever. The proposal in my mind takes too much of a product management or
> marketing view (perhaps knowingly). Here's how it will look the buttons one
> will push, etc..
>
> Totally knowingly, intentional even.
>
> > I think the "what" and "how it works" are important to decide on first,
> .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs
> for that, the "what" is much clearer.
>
> This seems contradictory to your previous statement. My document started
> the "what" and "how it works" discussion just fine. Whatever is unclear
> needs to be resolved *before* we jump into any implementation.
>
>
> > Looking over the efforts to date, couchdb-lucene, and geocouch, these
> two are quite different in terms of design, one is roughly loosely coupled,
> the other more native (in the same VM).
>
> Yup, we need to define how this fits into the plugin system. Maybe we
> never to something like couchdb-lucene, maybe we do native plugins first,
> and external plugins later, or the other way around. Thanks for making this
> more explicit, I will add this to the document.
>
>
> > A plugin architecture, in my mind, should emerge from the code
> refactoring and layout we're currently discussing.
>
> I respectfully disagree. I would like to start from the user and work my
> way down. What ever internal refactorings make sense to support the
> use-case, we will have to make. I trust that we are smart enough to make
> this in a way that is favourable to the rest of the code base.
>
> I definitely want to stress that I’d like to define the plugin feature
> from the user down, not from the technology up. This doesn’t mean we’ll
> bend over backwards to accommodate some crazy concept we draw up, but I
> think it helps keeping in mind who we do this for is great for making
> informed decisions rather than what’s easier for the code we have today.
>
>
> > We should ask and answer questions such as:
> >
> > 1. the role of externals
> >
> > 2 access to the storage layer (API?)
> >
> > 3. separation from http layer
> >
> > 4. support for code upgrades
> >
> > 5. balancing of resources
> >
> > I didn't mention Auth and Logging as I think these are separate in terms
> of concerns, more system oriented. Whereas geocouch and couchdb-lucene are
> actually extending the functionality in meaningful ways.
>
> Excellent observations and points!
>
> Cheers
> Jan
> --
>
> >
> >
> > On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
> >
> >> +1 - keep it simple for the first iteration.
> >>
> >>
> >> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
> >>
> >>> I quite like the rabbitmq approach (a lot like the apache httpd
> approach).
> >>> you can enable/disable/list plugins but the tool doesn't include the
> >>> downloading and managed repository bits, which is a huge rabbit hole
> (pun
> >>> intended).
> >>
> >>
> >>
> >
>
>
Re: CouchDB Plugins First Draft
Posted by Jan Lehnardt <ja...@apache.org>.
On Nov 1, 2012, at 11:01 , Bob Dionne <di...@dionne-associates.com> wrote:
> Reminds me of my favorite book - "Sketches of an Elephant"
>
> Jan, thanks for putting a stake in the ground, I've wanted to see this forever. The proposal in my mind takes too much of a product management or marketing view (perhaps knowingly). Here's how it will look the buttons one will push, etc..
Totally knowingly, intentional even.
> I think the "what" and "how it works" are important to decide on first, .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs for that, the "what" is much clearer.
This seems contradictory to your previous statement. My document started the "what" and "how it works" discussion just fine. Whatever is unclear needs to be resolved *before* we jump into any implementation.
> Looking over the efforts to date, couchdb-lucene, and geocouch, these two are quite different in terms of design, one is roughly loosely coupled, the other more native (in the same VM).
Yup, we need to define how this fits into the plugin system. Maybe we never to something like couchdb-lucene, maybe we do native plugins first, and external plugins later, or the other way around. Thanks for making this more explicit, I will add this to the document.
> A plugin architecture, in my mind, should emerge from the code refactoring and layout we're currently discussing.
I respectfully disagree. I would like to start from the user and work my way down. What ever internal refactorings make sense to support the use-case, we will have to make. I trust that we are smart enough to make this in a way that is favourable to the rest of the code base.
I definitely want to stress that I’d like to define the plugin feature from the user down, not from the technology up. This doesn’t mean we’ll bend over backwards to accommodate some crazy concept we draw up, but I think it helps keeping in mind who we do this for is great for making informed decisions rather than what’s easier for the code we have today.
> We should ask and answer questions such as:
>
> 1. the role of externals
>
> 2 access to the storage layer (API?)
>
> 3. separation from http layer
>
> 4. support for code upgrades
>
> 5. balancing of resources
>
> I didn't mention Auth and Logging as I think these are separate in terms of concerns, more system oriented. Whereas geocouch and couchdb-lucene are actually extending the functionality in meaningful ways.
Excellent observations and points!
Cheers
Jan
--
>
>
> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
>
>> +1 - keep it simple for the first iteration.
>>
>>
>> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>>
>>> I quite like the rabbitmq approach (a lot like the apache httpd approach).
>>> you can enable/disable/list plugins but the tool doesn't include the
>>> downloading and managed repository bits, which is a huge rabbit hole (pun
>>> intended).
>>
>>
>>
>
Re: CouchDB Plugins First Draft
Posted by Benoit Chesneau <bc...@gmail.com>.
On Thu, Nov 1, 2012 at 11:17 AM, Benoit Chesneau <bc...@gmail.com>wrote:
>
> Imo when we speak about plugins we should have in mind the of their
> deployement.
>
+ simplicity
Re: CouchDB Plugins First Draft
Posted by Jan Lehnardt <ja...@apache.org>.
Benoit,
Thanks a lot!
You bring up a lot of great material to the discussion along with your
expertise in writing and handling plugins in rcouch and related projects.
I’ll comb through this thread and extract all relevant information into
the gist/wiki document.
This is all great stuff! :)
Jan
--
On Nov 1, 2012, at 11:17 , Benoit Chesneau <bc...@gmail.com> wrote:
> there are also other plugins around :
>
> https://github.com/benoitc/couch_randomdoc
>
> https://github.com/benoitc/couch_zmq wich is using erlzmq
>
> https://github.com/benoitc/couch_es
>
> https://github.com/refuge/bigcouch_spatial
>
> https://github.com/refuge/couch_dbupdates
>
> https://github.com/ocastalabs/CouchDB-XO_Auth
>
> ....
>
>
> So on erlang the situation is quite complex. As a free reminder when we
> spoke about rabbitmq plugins in january, nobody wanted that which let me
> perplex. Like I said do we we want a plugin system or just a way to load
> app?
>
> About couchdb-lucene for example it could be rewritten to use more of the
> internals of couchdb. And probably using jinterface instead of living
> externally so it could be better integrated. (This is just an example of
> what could be done) . couch_es could be fully written in elixir or js and
> distributed as an internal couchapp. Same for random doc etc .
>
> Imo when we speak about plugins we should have in mind the of their
> deployement. This is one thing to install a plugin on a machine , another
> to make sure that this plugin is installed on others machines. And when I
> say other machines I'm not only speaking about machines in your lan (some
> of us have tools and people for that) but machines of remote possibly
> stranger people replicating your data . ANd this part is definitely not
> possible easily with something like rabbitmq plugins.
>
>
> - benoît
>
>
>
> On Thu, Nov 1, 2012 at 11:01 AM, Bob Dionne <di...@dionne-associates.com>wrote:
>
>> Reminds me of my favorite book - "Sketches of an Elephant"
>>
>> Jan, thanks for putting a stake in the ground, I've wanted to see this
>> forever. The proposal in my mind takes too much of a product management or
>> marketing view (perhaps knowingly). Here's how it will look the buttons one
>> will push, etc..
>>
>> I think the "what" and "how it works" are important to decide on first,
>> .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs
>> for that, the "what" is much clearer.
>>
>> Looking over the efforts to date, couchdb-lucene, and geocouch, these two
>> are quite different in terms of design, one is roughly loosely coupled, the
>> other more native (in the same VM). A plugin architecture, in my mind,
>> should emerge from the code refactoring and layout we're currently
>> discussing. We should ask and answer questions such as:
>>
>> 1. the role of externals
>>
>> 2 access to the storage layer (API?)
>>
>> 3. separation from http layer
>>
>> 4. support for code upgrades
>>
>> 5. balancing of resources
>>
>> I didn't mention Auth and Logging as I think these are separate in terms
>> of concerns, more system oriented. Whereas geocouch and couchdb-lucene are
>> actually extending the functionality in meaningful ways.
>>
>>
>> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
>>
>>> +1 - keep it simple for the first iteration.
>>>
>>>
>>> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>>>
>>>> I quite like the rabbitmq approach (a lot like the apache httpd
>> approach).
>>>> you can enable/disable/list plugins but the tool doesn't include the
>>>> downloading and managed repository bits, which is a huge rabbit hole
>> (pun
>>>> intended).
>>>
>>>
>>>
>>
>>
Re: CouchDB Plugins First Draft
Posted by Benoit Chesneau <bc...@gmail.com>.
there are also other plugins around :
https://github.com/benoitc/couch_randomdoc
https://github.com/benoitc/couch_zmq wich is using erlzmq
https://github.com/benoitc/couch_es
https://github.com/refuge/bigcouch_spatial
https://github.com/refuge/couch_dbupdates
https://github.com/ocastalabs/CouchDB-XO_Auth
....
So on erlang the situation is quite complex. As a free reminder when we
spoke about rabbitmq plugins in january, nobody wanted that which let me
perplex. Like I said do we we want a plugin system or just a way to load
app?
About couchdb-lucene for example it could be rewritten to use more of the
internals of couchdb. And probably using jinterface instead of living
externally so it could be better integrated. (This is just an example of
what could be done) . couch_es could be fully written in elixir or js and
distributed as an internal couchapp. Same for random doc etc .
Imo when we speak about plugins we should have in mind the of their
deployement. This is one thing to install a plugin on a machine , another
to make sure that this plugin is installed on others machines. And when I
say other machines I'm not only speaking about machines in your lan (some
of us have tools and people for that) but machines of remote possibly
stranger people replicating your data . ANd this part is definitely not
possible easily with something like rabbitmq plugins.
- benoît
On Thu, Nov 1, 2012 at 11:01 AM, Bob Dionne <di...@dionne-associates.com>wrote:
> Reminds me of my favorite book - "Sketches of an Elephant"
>
> Jan, thanks for putting a stake in the ground, I've wanted to see this
> forever. The proposal in my mind takes too much of a product management or
> marketing view (perhaps knowingly). Here's how it will look the buttons one
> will push, etc..
>
> I think the "what" and "how it works" are important to decide on first,
> .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs
> for that, the "what" is much clearer.
>
> Looking over the efforts to date, couchdb-lucene, and geocouch, these two
> are quite different in terms of design, one is roughly loosely coupled, the
> other more native (in the same VM). A plugin architecture, in my mind,
> should emerge from the code refactoring and layout we're currently
> discussing. We should ask and answer questions such as:
>
> 1. the role of externals
>
> 2 access to the storage layer (API?)
>
> 3. separation from http layer
>
> 4. support for code upgrades
>
> 5. balancing of resources
>
> I didn't mention Auth and Logging as I think these are separate in terms
> of concerns, more system oriented. Whereas geocouch and couchdb-lucene are
> actually extending the functionality in meaningful ways.
>
>
> On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
>
> > +1 - keep it simple for the first iteration.
> >
> >
> > On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
> >
> >> I quite like the rabbitmq approach (a lot like the apache httpd
> approach).
> >> you can enable/disable/list plugins but the tool doesn't include the
> >> downloading and managed repository bits, which is a huge rabbit hole
> (pun
> >> intended).
> >
> >
> >
>
>
Re: CouchDB Plugins First Draft
Posted by Bob Dionne <di...@dionne-associates.com>.
Reminds me of my favorite book - "Sketches of an Elephant"
Jan, thanks for putting a stake in the ground, I've wanted to see this forever. The proposal in my mind takes too much of a product management or marketing view (perhaps knowingly). Here's how it will look the buttons one will push, etc..
I think the "what" and "how it works" are important to decide on first, .eg. @rnewson's suggestion for something like RabbitMQ. Reading the docs for that, the "what" is much clearer.
Looking over the efforts to date, couchdb-lucene, and geocouch, these two are quite different in terms of design, one is roughly loosely coupled, the other more native (in the same VM). A plugin architecture, in my mind, should emerge from the code refactoring and layout we're currently discussing. We should ask and answer questions such as:
1. the role of externals
2 access to the storage layer (API?)
3. separation from http layer
4. support for code upgrades
5. balancing of resources
I didn't mention Auth and Logging as I think these are separate in terms of concerns, more system oriented. Whereas geocouch and couchdb-lucene are actually extending the functionality in meaningful ways.
On Nov 1, 2012, at 5:30 AM, Simon Metson <si...@cloudant.com> wrote:
> +1 - keep it simple for the first iteration.
>
>
> On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
>
>> I quite like the rabbitmq approach (a lot like the apache httpd approach).
>> you can enable/disable/list plugins but the tool doesn't include the
>> downloading and managed repository bits, which is a huge rabbit hole (pun
>> intended).
>
>
>
Re: CouchDB Plugins First Draft
Posted by Simon Metson <si...@cloudant.com>.
+1 - keep it simple for the first iteration.
On Wednesday, 31 October 2012 at 23:40, Robert Newson wrote:
> I quite like the rabbitmq approach (a lot like the apache httpd approach).
> you can enable/disable/list plugins but the tool doesn't include the
> downloading and managed repository bits, which is a huge rabbit hole (pun
> intended).