You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bloodhound.apache.org by Olemis Lang <ol...@gmail.com> on 2012/10/11 01:16:57 UTC
Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
On 10/10/12, Peter Koželj <pe...@digiverse.si> wrote:
>
[...]
>
> Just a few use cases for quick "does it float" test:
>
ok , challenge accepted
;)
>
> 1. Let's say we adopt some plugins for multiproducts, ticket relations,
> data import/export and so on...
When you say «adopt» I interpret that it means include them in
Bloodhound source code or install them by default . Otherwise , please
mention what is it about exactly .
<hr /> :P
Well , at the bottom of your message you ask about how to handle
inter-plugin dependencies in cases mentioned below . In general
there's no single magic solution to succeed but a set of tools , hacks
and practices . I'll write some simplistic answers below for each case
. I hope you understand that reality is more rich than anything I
might write in a few lines ...
> 2. Then we want to add user (admin) defined ticket relation per product
> 3. And then we want to support custom ticket workflows per product and per
> ticket type
afaics , the complexity here is related to the «per-product» modifier
. I'm writing right now about that subject in order to elaborate a bit
further the initial TracProposals [1]_ [2]_ [6]_ available on the
subject . In advance I tell you that (IMO) the level of uniformity and
decoupling needed in this particular case is not achieved by using any
other existing approach used ever before in Trac . So it requires a
bit of innovation ... fortunately that's what we are here for ;)
So I suggest you to talk about this once I finish those initial specs
, publish them , and we all analyze it ... It's just that as you can
see I've been working 100% these days and had no time yet to get this
done .
> 4. And now we want the ticket import/export from/to Jira to work across
> Trac and all other plugins (including ticket relations, multiproducts and
> custom per product workflows plugins)
That's what Interface class in the component architecture is for . In
such a hypothetical case let's limit the discussion to relational SQL
DBs . There will be a component that performs actual import / export
(let's call it SqlMigrationSystem) an Interface (let's call it
ISqlScriptContributor) and multiple components implementing the later
(e.g. one for ticket relations, multiproducts , custom per product
workflows , users , wiki pages , ...) .
So how could all this work . When user decides to e.g. export all data
then SqlMigrationSystem relies on Trac core to enumerate all the
components implementing that interface . It does not really matter
what plugin they belong to . For each one it invokes a method (defined
in the interface , so it works like a contract ;) so that each
implementor will fill a well-known data structure (e.g. like the ones
in bd_default.py but might be different if you want ...) in order to
describe the tables and associated data . After doing so
SqlMigrationSystem e.g. outputs a JSON, XML , CSV file ... whatever .
The plugin structure may be anything you want . Components may be
packaged in several ways and Trac component architecture will catch'em
all .
Disclaimer : All this is just a figment of my imagination , used to
illustrate how an hypothetically similar scenario could work . So
please don't pay much attention to the details , focus instead on the
overall picture .
> 5. Add white labeling across Trac and all other plugins
TagsPlugin [3]_ ? works in a similar way
> 6. Implement new search query language that will understand tickets,
> products and relations
idem ... nonetheless there might be some complications in this case ,
but in the end all the pieces are broken down into components
implementing an interface and a coordinator using theme all for a
particular purpose .
> 7. Add REST API's for all of the above
Once again it's the same thing . Nonetheless in this particular case a
lot of economy can be made by following a fairly simple strategy
previously followed by TracGVizPlugin [5]_ (... for other purposes
i.e. implement data sources compatible with Google Visualization API
;) and also experimented in RestOnTracPlugin [4]_ , even if not
published yet ;) . It happens that most of XML-RPC handlers associated
to resources (e.g. tickets , wiki , ...) have an identifier and
provide the following methods
- getAll
- get
- delete
- update
- create
have you ever seen anything so RESTful ? ;)
So it's possible to implement a generic REST frontend and perform
automatic translations by mapping REST requests to exsiting RPC
handlers methods , thus reusing existing code .
[...]
>
> Now, all this plugins (products, relations, import, export, search, white
> labeling, new search...) will either have to know filthy little details
> about each other
yes . they only need to know what interface to implement in order to
something they need to do
> (complex inter-plugin dependency)
not necessarily ... if you are familiar with inversion of control ,
aspect oriented programming and similar technologies you'll realize
that this conclusion is not always right . Indeed ExtensionOption may
be seen as some kind of inversion of control mechanism ... I don't
recall if there is a supported generic way to do these kinds of things
(i.e. IoC containers) in Trac , but I've never needed such bloody
monsters ...
> or some super heavy
> shifting in the plugin interface will be necessary.
That's what ITemplateStreamFilter interface is for . E.g. ticket
dependency plugins rely on this interface to insert information
(dependency UI) in existing web pages . And it all follows the same
principle mentioned above .
Aren't Trac & Genshi lovely ?
> And that super heavy
> shifting will have to be pushed back to Trac (if it wants it)
... nope ... no need to
> and only then
> to these plugins
... no ...
> or create new Bloodhound plugin API on top of Trac plugin
> API and then request plugin authors to support it.
>
we should avoid doing these kinds of things .
> Am I missing something?
>
only you can tell ... ;)
we are hanging out around here if you have further questions . always welcome .
.. [1] Project labels
(http://trac.edgewall.org/wiki/TracDev/Proposals/ProjectLabels)
.. [2] Multiple Projects within a Single Trac Environment
(http://trac.edgewall.org/wiki/TracMultipleProjects/SingleEnvironment)
.. [3] TagsPlugin
(http://trac-hacks.org/wiki/TagsPlugin)
.. [4] Publish Trac data via a REST-ful interface
(http://trac-hacks.org/wiki/RestOnTracPlugin)
.. [5] Integrate Trac with Google Visualization API
(http://trac-hacks.org/wiki/TracGvizPlugin)
.. [6] Trac Relations
(http://trac.edgewall.org/wiki/TracDev/Proposals/TracRelations)
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Olemis Lang <ol...@gmail.com>.
On 10/10/12, Branko Čibej <br...@wandisco.com> wrote:
> On 11.10.2012 05:49, Olemis Lang wrote:
>>> None of the above is a showstopper, but the interlocked dependencies
>>> will make managing feature-stable Bloodhound releases a bit like
>>> juggling eggs. It can be done, but you'd better not drop one. :)
>>>
>> well ... that has already happened . Trac-devs changed component
>> architecture slightly in 1.0 . Hence ThemeEnginePlugin didn't work
>> anymore . So we submitted a patch to t-h.o issue tracker .
>> ThemeEnginePlugin has not been updated since 2 years ago , nobody
>> cared . We managed to patch our own copy of Trac to make it work
>> without requiring any changes in ThemeEnginePlugin . That's exactly
>> what we are all using nowadays , otherwise the new design would not be
>> a reality . I requested maintainership of ThemeEnginePlugin days ago
>> and I'm in charge now ... so I don't see the storm coming yet .
>
> /You/ requested maintainership. How does that help Apache Bloodhound if,
> two weeks or two years from now, you find a new interest in life and
> stop maintaining it?
>
The same will happen over and over . If there's real interest someone
else will request to adopt the plugin , ask me , I say «it's ok
fellows , I'm devoted to fishing virgin marlins now, Trac no more ...»
, and that's it .
> The issue I see here is that, essentially, the Apache Bloodhound
> community doesn't have any real control over some of the core components
> of Bloodhound-the-product.
I don't see things that way . What I'm trying to explain is that
1. we have *a lot* of options at hand to handle collaboration with
Trac-dev and Trac-Hacks
2. if we'd like to «have real control over all the core components
of Bloodhound-the-product» then we can , because we'll always
be able to fork the core itself , and any other Trac hacks we might
need to make things happen , as long as we select them carefully ,
which is ok for MIT and BSD licenses .
3. ... nevertheless it's important to collaborate with them
(/me included ;) and establish bi-directional relationships .
4. ... and nothing fatal has happened so far
> That's nothing new in either Apache or open
> source in general; the difference is that BH has so /many/ mandatory
> external dependencies for very fundamental, core functionality.
>
Read 2. above . That's an obvious consequence of the fact that
Trac-dev and Trac-Hacks have been spending quite many years building
features and plugins , and this project decided to take advantage of
it . Otherwise we could just spend some years rewriting everything
they've done up to this point .
> It's not an engineering problem, it's a management problem; and
> open-source projects are notoriously bad at handling complex management
> problems.
>
... especially when it comes to pluggable architectures , like Trac's .
> I guess I'm just worried about the long-term stability and viability of
> the project.
>
TBH I understand , but if you are aware of all the options at hand and
still think this way it's maybe because you have something different
in mind . So what is it about ? Trac-dev joining the ASF ? It seems
that won't happen any time soon . Anything else ?
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Peter Koželj <pe...@digiverse.si>.
Agree,
Brane managed to put in words what has been troubling me and was not able
to pinpoint it exactly: "It's not an engineering problem, it's a management
problem;"
And somebody will have to do it, end we are all engineers :)
Peter
On Thu, Oct 11, 2012 at 6:19 AM, Branko Čibej <br...@wandisco.com> wrote:
> On 11.10.2012 05:49, Olemis Lang wrote:
> >> None of the above is a showstopper, but the interlocked dependencies
> >> will make managing feature-stable Bloodhound releases a bit like
> >> juggling eggs. It can be done, but you'd better not drop one. :)
> >>
> > well ... that has already happened . Trac-devs changed component
> > architecture slightly in 1.0 . Hence ThemeEnginePlugin didn't work
> > anymore . So we submitted a patch to t-h.o issue tracker .
> > ThemeEnginePlugin has not been updated since 2 years ago , nobody
> > cared . We managed to patch our own copy of Trac to make it work
> > without requiring any changes in ThemeEnginePlugin . That's exactly
> > what we are all using nowadays , otherwise the new design would not be
> > a reality . I requested maintainership of ThemeEnginePlugin days ago
> > and I'm in charge now ... so I don't see the storm coming yet .
>
> /You/ requested maintainership. How does that help Apache Bloodhound if,
> two weeks or two years from now, you find a new interest in life and
> stop maintaining it?
>
> The issue I see here is that, essentially, the Apache Bloodhound
> community doesn't have any real control over some of the core components
> of Bloodhound-the-product. That's nothing new in either Apache or open
> source in general; the difference is that BH has so /many/ mandatory
> external dependencies for very fundamental, core functionality.
>
> It's not an engineering problem, it's a management problem; and
> open-source projects are notoriously bad at handling complex management
> problems.
>
> I guess I'm just worried about the long-term stability and viability of
> the project.
>
> > <joke>
> > Titanium eggs never break , sir
> > :D
> > </joke>
>
> But they sure can get squashed. :-P
>
> -- Brane
>
> --
> Certified & Supported Apache Subversion Downloads:
> http://www.wandisco.com/subversion/download
>
>
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Branko Čibej <br...@wandisco.com>.
On 11.10.2012 05:49, Olemis Lang wrote:
>> None of the above is a showstopper, but the interlocked dependencies
>> will make managing feature-stable Bloodhound releases a bit like
>> juggling eggs. It can be done, but you'd better not drop one. :)
>>
> well ... that has already happened . Trac-devs changed component
> architecture slightly in 1.0 . Hence ThemeEnginePlugin didn't work
> anymore . So we submitted a patch to t-h.o issue tracker .
> ThemeEnginePlugin has not been updated since 2 years ago , nobody
> cared . We managed to patch our own copy of Trac to make it work
> without requiring any changes in ThemeEnginePlugin . That's exactly
> what we are all using nowadays , otherwise the new design would not be
> a reality . I requested maintainership of ThemeEnginePlugin days ago
> and I'm in charge now ... so I don't see the storm coming yet .
/You/ requested maintainership. How does that help Apache Bloodhound if,
two weeks or two years from now, you find a new interest in life and
stop maintaining it?
The issue I see here is that, essentially, the Apache Bloodhound
community doesn't have any real control over some of the core components
of Bloodhound-the-product. That's nothing new in either Apache or open
source in general; the difference is that BH has so /many/ mandatory
external dependencies for very fundamental, core functionality.
It's not an engineering problem, it's a management problem; and
open-source projects are notoriously bad at handling complex management
problems.
I guess I'm just worried about the long-term stability and viability of
the project.
> <joke>
> Titanium eggs never break , sir
> :D
> </joke>
But they sure can get squashed. :-P
-- Brane
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Branko Čibej <br...@wandisco.com>.
On 11.10.2012 05:49, Olemis Lang wrote:
> On 10/10/12, Branko Čibej <br...@wandisco.com> wrote:
>> On 11.10.2012 01:16, Olemis Lang wrote:
>>> On 10/10/12, Peter Koželj <pe...@digiverse.si> wrote:
>>>> 4. And now we want the ticket import/export from/to Jira to work across
>>>> Trac and all other plugins (including ticket relations, multiproducts
>>>> and
>>>> custom per product workflows plugins)
>>> That's what Interface class in the component architecture is for . In
>>> such a hypothetical case let's limit the discussion to relational SQL
>>> DBs . There will be a component that performs actual import / export
>>> (let's call it SqlMigrationSystem) an Interface (let's call it
>>> ISqlScriptContributor) and multiple components implementing the later
>>> (e.g. one for ticket relations, multiproducts , custom per product
>>> workflows , users , wiki pages , ...) .
>>>
>>> So how could all this work . When user decides to e.g. export all data
>>> then SqlMigrationSystem relies on Trac core to enumerate all the
>>> components implementing that interface . It does not really matter
>>> what plugin they belong to .
>> Etc. Precisely the point. Every plugin that Bloodhound uses for what it
>> considers core functionality must implement this interface, or the whole
>> export/import story falls on its face.
> Well , that's exactly the point , isn't it ? Build a generic framework
> and give components some room to make it more specific in many
> different ways
> ;)
>
> ... maybe the interface is part of Trac itself ...
>
>> Since these are plugins that the
>> Apache Bloodhound project does not control, you either have to ask the
>> plugin authors very nicely to implement that interface,
> step 1 .
>
>> or do it
>> yourself and hope your patches are accepted,
> step 2
>
>> or maintain a perpetual
>> stack of patches for every core plugin,
> that's what we do with Trac vendor branch already , yes ...
>
>> or fork the plugin (for which
>> you need a code grant to relicense under ALv2, which ain't gonna happen
>> if you couldn't arrange any of the other options with the author),
> That depends on the original license . BSD and MIT should not be problematic ...
Have you read the ASF legal requirements for releases? :)
-- Brane
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Olemis Lang <ol...@gmail.com>.
On 10/10/12, Branko Čibej <br...@wandisco.com> wrote:
> On 11.10.2012 01:16, Olemis Lang wrote:
>> On 10/10/12, Peter Koželj <pe...@digiverse.si> wrote:
>>> 4. And now we want the ticket import/export from/to Jira to work across
>>> Trac and all other plugins (including ticket relations, multiproducts
>>> and
>>> custom per product workflows plugins)
>> That's what Interface class in the component architecture is for . In
>> such a hypothetical case let's limit the discussion to relational SQL
>> DBs . There will be a component that performs actual import / export
>> (let's call it SqlMigrationSystem) an Interface (let's call it
>> ISqlScriptContributor) and multiple components implementing the later
>> (e.g. one for ticket relations, multiproducts , custom per product
>> workflows , users , wiki pages , ...) .
>>
>> So how could all this work . When user decides to e.g. export all data
>> then SqlMigrationSystem relies on Trac core to enumerate all the
>> components implementing that interface . It does not really matter
>> what plugin they belong to .
>
> Etc. Precisely the point. Every plugin that Bloodhound uses for what it
> considers core functionality must implement this interface, or the whole
> export/import story falls on its face.
Well , that's exactly the point , isn't it ? Build a generic framework
and give components some room to make it more specific in many
different ways
;)
... maybe the interface is part of Trac itself ...
> Since these are plugins that the
> Apache Bloodhound project does not control, you either have to ask the
> plugin authors very nicely to implement that interface,
step 1 .
> or do it
> yourself and hope your patches are accepted,
step 2
> or maintain a perpetual
> stack of patches for every core plugin,
that's what we do with Trac vendor branch already , yes ...
> or fork the plugin (for which
> you need a code grant to relicense under ALv2, which ain't gonna happen
> if you couldn't arrange any of the other options with the author),
That depends on the original license . BSD and MIT should not be problematic ...
> or
> write your own plugin providing the same functionality.
>
... well ... afaics you have many options at hand . I don't get it . I
thought you were saying you were locked-in somehow .
You want a tomato . You either buy the seeds , take care of the plants
make them grow , then harvest all of it and then eat it ... or just go
to the supermarket and buy some tomatoes . There's no almighty tomato
.
> None of the above is a showstopper, but the interlocked dependencies
> will make managing feature-stable Bloodhound releases a bit like
> juggling eggs. It can be done, but you'd better not drop one. :)
>
well ... that has already happened . Trac-devs changed component
architecture slightly in 1.0 . Hence ThemeEnginePlugin didn't work
anymore . So we submitted a patch to t-h.o issue tracker .
ThemeEnginePlugin has not been updated since 2 years ago , nobody
cared . We managed to patch our own copy of Trac to make it work
without requiring any changes in ThemeEnginePlugin . That's exactly
what we are all using nowadays , otherwise the new design would not be
a reality . I requested maintainership of ThemeEnginePlugin days ago
and I'm in charge now ... so I don't see the storm coming yet .
<joke>
Titanium eggs never break , sir
:D
</joke>
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Managing plugin dependencies WAS: Tests and CI (was: A word on
Release 2)
Posted by Branko Čibej <br...@wandisco.com>.
On 11.10.2012 01:16, Olemis Lang wrote:
> On 10/10/12, Peter Koželj <pe...@digiverse.si> wrote:
>> 4. And now we want the ticket import/export from/to Jira to work across
>> Trac and all other plugins (including ticket relations, multiproducts and
>> custom per product workflows plugins)
> That's what Interface class in the component architecture is for . In
> such a hypothetical case let's limit the discussion to relational SQL
> DBs . There will be a component that performs actual import / export
> (let's call it SqlMigrationSystem) an Interface (let's call it
> ISqlScriptContributor) and multiple components implementing the later
> (e.g. one for ticket relations, multiproducts , custom per product
> workflows , users , wiki pages , ...) .
>
> So how could all this work . When user decides to e.g. export all data
> then SqlMigrationSystem relies on Trac core to enumerate all the
> components implementing that interface . It does not really matter
> what plugin they belong to .
Etc. Precisely the point. Every plugin that Bloodhound uses for what it
considers core functionality must implement this interface, or the whole
export/import story falls on its face. Since these are plugins that the
Apache Bloodhound project does not control, you either have to ask the
plugin authors very nicely to implement that interface, or do it
yourself and hope your patches are accepted, or maintain a perpetual
stack of patches for every core plugin, or fork the plugin (for which
you need a code grant to relicense under ALv2, which ain't gonna happen
if you couldn't arrange any of the other options with the author), or
write your own plugin providing the same functionality.
None of the above is a showstopper, but the interlocked dependencies
will make managing feature-stable Bloodhound releases a bit like
juggling eggs. It can be done, but you'd better not drop one. :)
-- Brane
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download