You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by Kevin Menard <ni...@gmail.com> on 2009/04/11 00:27:50 UTC
maven plugin for importing DB schema
I finally have made some time to work on CAY-1029. This is a maven plugin
for importing DB schema and generating a datamap. While I do have some use
for this, others were more interested in the feature than me. So, I'm
looking at some feedback on it. As of now, the basics are supported with
new features expected. Once the maven plugin is completed, I'll look at
refactoring so that it can be used from ant as well.
Feedback would be much appreciated. Docs are in Cayenne:
http://cwiki.apache.org/confluence/display/CAYDOC/maven2-cdbimport
--
Kevin
Re: maven plugin for importing DB schema
Posted by Kevin Menard <ni...@gmail.com>.
Hi Adrian,
I don't think anyone is disagreeing with the value in either a unified
codebase or in automation tools. It's simple matter of resources. We have
about a half dozen active developers at any given time of which a percentage
of that is working on code. While the project has its priorities, so do
individual developers. I know that may sound odd, but it's actually very
hard to motivate someone to work in their free time on something that may
not directly benefit them. In my current situation I do no commercial work
with Cayenne in any capacity -- I work on it because I like the project.
So, as things are a volunteer effort the choices tend to be one of three:
- accept that a given feature may not added quickly (if at all)
- try to work on the feature yourself (the dev team is always willing to
help)
- consider sponsoring development on a feature (this has worked in the past)
As it stands with the DB import tools, we are firmly in the first of those
three. I'd be happy to work with you on the second point if you'd like.
And if you'd rather sponsor a feature, we can explore that avenue as well.
As you can see, the bullet points also plot a natural continuum between
cost for me to cost for you, in terms of development time, development
capital, and completion time.
--
Kevin
On Sat, Apr 11, 2009 at 10:23 AM, Adrian A. <a....@googlemail.com>wrote:
> >
> > I'm not sure the duplicate work is coming from. No one else has picked
> up
> > the issues.
> >
> Well, from the user's perspective, the issues are one interesting and
> somewhat nontransparent thing, i.e. many issues seem too big to be
> implemented by the community (I suppose in a commercial project the issue
> would be broken into several small sub-issues that are easily doable). So
> just the pure assignment of an issue doesn't show how things are going, so
> that others users to pick small parts of it on the way and help.
>
> It's also not as simple as just refactoring the modeler code. DbLoader
> > requires a delegate that handles the addition and removal of object and
> DB
> > entities and currently there's a lot of code tied into the UI from that
> > delegate.
>
> I saw that it's quite complicated for what is supposed to do (but this
> might
> be simply because I'm not used to the code, or was expecting something else
> :) ).
>
>
> > I'm not convinced that abstracting all of that out is going to
> > make the code any clearer.
>
> I still think this should be the way - the entire reverse engineering
> process should be UI independent so that there aren't 3 different
> implementation with 3 different places to fix bugs or do improvements.
> Also IntelliJ is very smart at helping with refactorings (I see that your
> project already has free licenses for it)
>
> I think, the success of the Cayenne (and Modeler) depends to great degree
> on
> how smart is the reverse engineering step - the smarter it is, the less
> manual work has the user to do, so the greater the user satisfaction is.
> With scenarios of over 50 tables, manual intervention is simply a pain
> (from
> what I experienced so far :( ).
>
>
> > So, I'm tackling this on a one-off basis first,
> > seeing what's common, and then taking it from there. Given that I drove
> > primary development on all of the maven plugins and a large part of the
> ant
> > ones, I feel this is the best approach to take.
> >
> > If you'd like to help out testing CAY-1029, feel free to start up a
> simple
> > project in maven. If not, that's fine, too. I'll get to CAY-1197
> someday
> > and I'll set up a simple project in ant. There are people interested in
> > both so I should get feedback one way or another.
> >
> > Incidentally, the maven plugins and ant tasks are a great place for a
> > community member to supply a patch. In fact, it's how I got started with
> > Cayenne. They're abstracted enough out of core that the opportunity for
> > damage is fairly minimal.
>
> I will give it a try.
>
> I think however that to the same category of "smart reverse enginnering"
> should be these too:
> #1. https://issues.apache.org/jira/browse/CAY-154
> (better said the UI independent code - i.e. to detect automatically
> many2many and flattern where appropiate - if a setting for this is active)
> #2. https://issues.apache.org/jira/browse/CAY-400
> (to be able to get the comments from the DB attached to the object and DB
> entites)
> #3. https://issues.apache.org/jira/browse/CAY-209
> #4. https://issues.apache.org/jira/browse/CAY-850
> (or something similar - i.e. where there's a "name matching" but no
> concrete
> foreign key, to add a single directed relationship. E.g. many tables have
> "created_by_user_id", but no FK to the user)
> #5 Singularize of table names (if a setting is active)
>
> As you can see, for many of the above feature requests, a seprated and UI
> independent reverse engineering step would allow to make it happen quite
> faster (since it would be done and tested in a single place - the rest: UI,
> ANT, and Maven being just "wrappers" :) )
>
> Thank you,
>
> A.
>
Re: maven plugin for importing DB schema
Posted by "Adrian A." <a....@googlemail.com>.
>
> I'm not sure the duplicate work is coming from. No one else has picked up
> the issues.
>
Well, from the user's perspective, the issues are one interesting and
somewhat nontransparent thing, i.e. many issues seem too big to be
implemented by the community (I suppose in a commercial project the issue
would be broken into several small sub-issues that are easily doable). So
just the pure assignment of an issue doesn't show how things are going, so
that others users to pick small parts of it on the way and help.
It's also not as simple as just refactoring the modeler code. DbLoader
> requires a delegate that handles the addition and removal of object and DB
> entities and currently there's a lot of code tied into the UI from that
> delegate.
I saw that it's quite complicated for what is supposed to do (but this might
be simply because I'm not used to the code, or was expecting something else
:) ).
> I'm not convinced that abstracting all of that out is going to
> make the code any clearer.
I still think this should be the way - the entire reverse engineering
process should be UI independent so that there aren't 3 different
implementation with 3 different places to fix bugs or do improvements.
Also IntelliJ is very smart at helping with refactorings (I see that your
project already has free licenses for it)
I think, the success of the Cayenne (and Modeler) depends to great degree on
how smart is the reverse engineering step - the smarter it is, the less
manual work has the user to do, so the greater the user satisfaction is.
With scenarios of over 50 tables, manual intervention is simply a pain (from
what I experienced so far :( ).
> So, I'm tackling this on a one-off basis first,
> seeing what's common, and then taking it from there. Given that I drove
> primary development on all of the maven plugins and a large part of the ant
> ones, I feel this is the best approach to take.
>
> If you'd like to help out testing CAY-1029, feel free to start up a simple
> project in maven. If not, that's fine, too. I'll get to CAY-1197 someday
> and I'll set up a simple project in ant. There are people interested in
> both so I should get feedback one way or another.
>
> Incidentally, the maven plugins and ant tasks are a great place for a
> community member to supply a patch. In fact, it's how I got started with
> Cayenne. They're abstracted enough out of core that the opportunity for
> damage is fairly minimal.
I will give it a try.
I think however that to the same category of "smart reverse enginnering"
should be these too:
#1. https://issues.apache.org/jira/browse/CAY-154
(better said the UI independent code - i.e. to detect automatically
many2many and flattern where appropiate - if a setting for this is active)
#2. https://issues.apache.org/jira/browse/CAY-400
(to be able to get the comments from the DB attached to the object and DB
entites)
#3. https://issues.apache.org/jira/browse/CAY-209
#4. https://issues.apache.org/jira/browse/CAY-850
(or something similar - i.e. where there's a "name matching" but no concrete
foreign key, to add a single directed relationship. E.g. many tables have
"created_by_user_id", but no FK to the user)
#5 Singularize of table names (if a setting is active)
As you can see, for many of the above feature requests, a seprated and UI
independent reverse engineering step would allow to make it happen quite
faster (since it would be done and tested in a single place - the rest: UI,
ANT, and Maven being just "wrappers" :) )
Thank you,
A.
Re: maven plugin for importing DB schema
Posted by Kevin Menard <ni...@gmail.com>.
I'm not sure the duplicate work is coming from. No one else has picked up
the issues.
It's also not as simple as just refactoring the modeler code. DbLoader
requires a delegate that handles the addition and removal of object and DB
entities and currently there's a lot of code tied into the UI from that
delegate. I'm not convinced that abstracting all of that out is going to
make the code any clearer. So, I'm tackling this on a one-off basis first,
seeing what's common, and then taking it from there. Given that I drove
primary development on all of the maven plugins and a large part of the ant
ones, I feel this is the best approach to take.
If you'd like to help out testing CAY-1029, feel free to start up a simple
project in maven. If not, that's fine, too. I'll get to CAY-1197 someday
and I'll set up a simple project in ant. There are people interested in
both so I should get feedback one way or another.
Incidentally, the maven plugins and ant tasks are a great place for a
community member to supply a patch. In fact, it's how I got started with
Cayenne. They're abstracted enough out of core that the opportunity for
damage is fairly minimal.
--
Kevin
On Sat, Apr 11, 2009 at 6:15 AM, Adrian A. <a....@googlemail.com>wrote:
> > Second guessing the developer priority is not going to help here.
>
> It might help however other users trying give some help and also avoid
> duplicate work.
>
> A.
>
Re: maven plugin for importing DB schema
Posted by "Adrian A." <a....@googlemail.com>.
> Second guessing the developer priority is not going to help here.
It might help however other users trying give some help and also avoid
duplicate work.
A.
Re: maven plugin for importing DB schema
Posted by Andrus Adamchik <an...@objectstyle.org>.
How about this - Kevin did it because (1) it improves Cayenne and (2)
he felt like doing it. Every day we hear "I am using X, so X has to be
the most popular technology out there, so you must make it your
highest priority to integrate with X"... Fine, except that "X" is
different for every person, and at the end we have the entire alphabet
to take care of.
Technically both Maven and Ant plugins are trivial, as there is a
class in Cayenne core called DbLoader. However doing a production
quality work means writing the code, unit tests and documentation.
This means someone spending more than a few minutes on that.
Everything can't happen overnight. Second guessing the developer
priority is not going to help here.
Andrus
On Apr 11, 2009, at 12:45 PM, Adrian A. wrote:
>>
>> Well, please understand that I'm writing this not because I need it
>> but
>> because others have requested it and I can see value in having it.
>> At the
>> moment, all my projects are in maven, so it made it a lot easier to
>> test
>> that way. Frankly, I'd just assume write in jRuby and use buildr.
>> The
>> process would be a lot simpler for me.
>
> Please feel free to correct me if I'm wrong, but:
> - shouldn't exactly the same functionality be in Cayenne Modeler,
> the Maven
> and the ANT task regarding the reverse engineering process? I.e.
> from the
> same user input parameters to produce exactly the same output - aka
> a nice
> and correctly generated datamap?
> - Cayenne Modeler already does this reverse engineering, so the
> functionality is practically already there?
> - the Maven and the ANT task would be than just a wrapper to call that
> functionality?
> - this would be than similar to the case of Cayenne ClassGeneration?
>
> If so, than wouldn't be the first step to refactor the actual Cayenne
> Modeler functionality(the already working) that already does reverse
> engineering to some common classes in:
> org.apache.cayenne.gen.*
> like it was done for ClassGeneration?
> to be simply callable from the Maven and ANT task, thus making them
> trivial
> to implement?
>
> Thank you,
> A.
Re: maven plugin for importing DB schema
Posted by "Adrian A." <a....@googlemail.com>.
>
> Well, please understand that I'm writing this not because I need it but
> because others have requested it and I can see value in having it. At the
> moment, all my projects are in maven, so it made it a lot easier to test
> that way. Frankly, I'd just assume write in jRuby and use buildr. The
> process would be a lot simpler for me.
Please feel free to correct me if I'm wrong, but:
- shouldn't exactly the same functionality be in Cayenne Modeler, the Maven
and the ANT task regarding the reverse engineering process? I.e. from the
same user input parameters to produce exactly the same output - aka a nice
and correctly generated datamap?
- Cayenne Modeler already does this reverse engineering, so the
functionality is practically already there?
- the Maven and the ANT task would be than just a wrapper to call that
functionality?
- this would be than similar to the case of Cayenne ClassGeneration?
If so, than wouldn't be the first step to refactor the actual Cayenne
Modeler functionality(the already working) that already does reverse
engineering to some common classes in:
org.apache.cayenne.gen.*
like it was done for ClassGeneration?
to be simply callable from the Maven and ANT task, thus making them trivial
to implement?
Thank you,
A.
Re: maven plugin for importing DB schema
Posted by Kevin Menard <ni...@gmail.com>.
Well, please understand that I'm writing this not because I need it but
because others have requested it and I can see value in having it. At the
moment, all my projects are in maven, so it made it a lot easier to test
that way. Frankly, I'd just assume write in jRuby and use buildr. The
process would be a lot simpler for me.
--
Kevin
On Fri, Apr 10, 2009 at 8:37 PM, Adrian A. <a....@googlemail.com>wrote:
>
>
> Why not start with the ANT version ?:
> https://issues.apache.org/jira/browse/CAY-1197
>
> Much more users could benefit from it (most users don't use maven), and you
> would get some real feedback much faster.
>
> Thank you,
> A.
>
Re: maven plugin for importing DB schema
Posted by "Adrian A." <a....@googlemail.com>.
>
> I finally have made some time to work on CAY-1029. This is a maven plugin
> for importing DB schema and generating a datamap. While I do have some use
> for this, others were more interested in the feature than me. So, I'm
> looking at some feedback on it. As of now, the basics are supported with
> new features expected. Once the maven plugin is completed, I'll look at
> refactoring so that it can be used from ant as well.
> Feedback would be much appreciated. Docs are in Cayenne:
>
> http://cwiki.apache.org/confluence/display/CAYDOC/maven2-cdbimport
Why not start with the ANT version ?:
https://issues.apache.org/jira/browse/CAY-1197
Much more users could benefit from it (most users don't use maven), and you
would get some real feedback much faster.
Thank you,
A.