Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

Hi folks.

I'm temporarily locked out of JIRA. I have some fixes to the maven plugin.

If infra unsticks me, I'll make a JIRA, but if not, would anyone be
disturbed if I made a jira-less commit?

The fixes are:

1: make the default output generated-sources/jena. Using
generated-sources tangles up with 'annotations'.

2: make the output directory configurable.

3: add the output directory to the source roots.

--benson

Re: Fixes to the schemagen plugin

[Andy Seaborne <an...@apache.org>]

benson,

We have a build failure:

https://builds.apache.org/user/andy/my-views/view/Jena/job/Jena_Development_Test/1311/

org.apache.maven:maven-project:2.2.1 is way out of date causes 
dependency convergence errors on:

org.apache.maven:maven-model
junit:junit
org.codehaus.plexus:plexus-utils

and others.

We run the enforcer plugin ... so the build breaks.

maven-project:2.2.1 was released in Aug 2009.


	Andy

On 27/06/14 03:35, Benson Margulies wrote:
> OK, got into JIRA, created two jiras, committed, marked them resolved.
>
> Andy, could you please add a component for the jena-maven-tools and a
> version for 0.5 and 0.6 and I'll spruce them up further.
>
> On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies <bi...@gmail.com> wrote:
>> Hi folks.
>>
>> I'm temporarily locked out of JIRA. I have some fixes to the maven plugin.
>>
>> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
>> disturbed if I made a jira-less commit?
>>
>> The fixes are:
>>
>> 1: make the default output generated-sources/jena. Using
>> generated-sources tangles up with 'annotations'.
>>
>> 2: make the output directory configurable.
>>
>> 3: add the output directory to the source roots.
>>
>> --benson

Re: Fixes to the schemagen plugin

[Ian Dickinson <i....@gmail.com>]

On Fri, Jun 27, 2014 at 2:21 PM, Benson Margulies <bi...@gmail.com> wrote:
> Once you all are comfy with the changes from last night
In addition, as I said in the Jira comment, I'm *un*comfy with
changing default output locations with zero notice. I can easily see
this breaking automated builds or other workflows. At the very least,
we should consult with users@ before making this change.

Ian

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

Ian et al,

I reverted the code changes pending discussion. I left the pom
changes; I assert that they are improvement.


On Fri, Jun 27, 2014 at 10:55 AM, Benson Margulies
<bi...@gmail.com> wrote:
> On Fri, Jun 27, 2014 at 10:47 AM, Ian Dickinson <i....@gmail.com> wrote:
>> On Fri, Jun 27, 2014 at 3:31 PM, Benson Margulies <bi...@gmail.com> wrote:
>>> On Fri, Jun 27, 2014 at 10:05 AM, Ian Dickinson <i....@gmail.com> wrote:
>>>> Actually, now that I've reviewed it, there's a conflict between this
>>>> patch and the current behaviour. At the moment, the way to set the
>>>> output directory to something other than target/generated-sources is
>>>> to set the output configuration parameter. If this is a directory,
>>>> generated files go into that directory. If that option is set in the
>>>> defaults section of the schemagen config, it affects all of the
>>>> generated files. Tl;dr: there is already a mechanism for changing the
>>>> default output directory for all generated files.
>>>>
>>>> Voting to reject this patch until it's clearer why users need two
>>>> mechanisms for achieving the same end.
>>>
>>> So, I'll remove the redundancy, but have I convinced you about the
>>> problem of it not getting compiled?
>> Yes, absolutely. I'm not using the plugin day-by-day (I rarely do Java
>> dev these days, in fact), so direct experience by someone using the
>> tool definitely takes precedence.
>>
>> I'm actually not sure what the right solution here is. With the
>> current mechanism, the output directory can be specified on an
>> vocabulary-by-vocabulary basis. Any of those locations might already
>> be in the source tree already (e.g. we support the scenario where
>> someone generates the .java file directly under src/main/java, so that
>> the translated file can be kept under source control). So I'm not sure
>> - and you are in a better position to advise on this - that just
>> calling addCompileSourceRoot() on each output directory would be
>> correct.
>
>
>
>
>>
>> Perhaps the answer is that the location of the output files should
>> remain with the current configuration process, but there should be an
>> additional means, via the pom.xml, to add one or more directories to
>> the sources root. That could have a default value, so *if* the default
>> default output is left as target/generated-sources/jena, and *if* the
>> user doesn't change anything in the pom, then the output Java files
>> will still compile.
>
> To do exactly what you suggest, all we need to do is revert my change;
> the build-helper-maven-plugin is already a way to add a directory to
> the source roots via pom config.
>
> However, I have a more complex idea to offer you:
>
> Maybe redundancy is not redundancy.
>
> We tell users: If you want to just produce sources to include in the
> build, you use my new top-level property as the single source root,
> and we will add it to the build, and all will be well. If you want to
> do something more complex, you can use the older scheme, and it's up
> to you to arrange for it to compile, perhaps using the
> build-helper-maven-plugin.
>
> To make this really painless, we'd need one more boolean property:
> useOutputDirectory. Default is false, preserving old behavior. Set to
> true, gets new behavior, respecting new property.
>
> WDYT?
>
>>
>> With respect to changing the default, if writing to
>> target/generated-sources simply doesn't work with Maven 3, then the
>> change of default will be less intrusive, because presumably people
>> are not doing that at the moment. I still think it would be worth
>> raising on users@, to see if anyone would be impacted.
>>
>> There's enough complexity here that it's probably worth extending the
>> current set of integration tests to cover this new scenario.
>>
>
> agreed:
>
>> Ian

Re: Fixes to the schemagen plugin

[Stephen Allen <sa...@apache.org>]

Seems to make sense.  I use schemagen and maven, but I don't use the
plugin.  Rather I use "exec-maven-plugin" combined with
"build-helper-maven-plugin" as described at [1].

-Stephen
[1] https://jena.apache.org/documentation/tools/schemagen.html


On Tuesday, July 1, 2014, Ian Dickinson <i....@gmail.com> wrote:

> On Fri, Jun 27, 2014 at 3:55 PM, Benson Margulies <bimargulies@gmail.com
> <javascript:;>> wrote:
> > To do exactly what you suggest, all we need to do is revert my change;
> > the build-helper-maven-plugin is already a way to add a directory to
> > the source roots via pom config.
> >
> > However, I have a more complex idea to offer you:
> >
> > Maybe redundancy is not redundancy.
> >
> > We tell users: If you want to just produce sources to include in the
> > build, you use my new top-level property as the single source root,
> > and we will add it to the build, and all will be well. If you want to
> > do something more complex, you can use the older scheme, and it's up
> > to you to arrange for it to compile, perhaps using the
> > build-helper-maven-plugin.
> >
> > To make this really painless, we'd need one more boolean property:
> > useOutputDirectory. Default is false, preserving old behavior. Set to
> > true, gets new behavior, respecting new property.
> >
> > WDYT?
> I can see your logic. OK, since you're the one with the active use
> case, and since no-one has commented at all on users@, I'm in favour
> of you proceeding as you suggest.
>
> Any one else?
>
> Ian
>

Re: Fixes to the schemagen plugin

[Ian Dickinson <i....@gmail.com>]

On Fri, Jun 27, 2014 at 3:55 PM, Benson Margulies <bi...@gmail.com> wrote:
> To do exactly what you suggest, all we need to do is revert my change;
> the build-helper-maven-plugin is already a way to add a directory to
> the source roots via pom config.
>
> However, I have a more complex idea to offer you:
>
> Maybe redundancy is not redundancy.
>
> We tell users: If you want to just produce sources to include in the
> build, you use my new top-level property as the single source root,
> and we will add it to the build, and all will be well. If you want to
> do something more complex, you can use the older scheme, and it's up
> to you to arrange for it to compile, perhaps using the
> build-helper-maven-plugin.
>
> To make this really painless, we'd need one more boolean property:
> useOutputDirectory. Default is false, preserving old behavior. Set to
> true, gets new behavior, respecting new property.
>
> WDYT?
I can see your logic. OK, since you're the one with the active use
case, and since no-one has commented at all on users@, I'm in favour
of you proceeding as you suggest.

Any one else?

Ian

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

On Fri, Jun 27, 2014 at 10:47 AM, Ian Dickinson <i....@gmail.com> wrote:
> On Fri, Jun 27, 2014 at 3:31 PM, Benson Margulies <bi...@gmail.com> wrote:
>> On Fri, Jun 27, 2014 at 10:05 AM, Ian Dickinson <i....@gmail.com> wrote:
>>> Actually, now that I've reviewed it, there's a conflict between this
>>> patch and the current behaviour. At the moment, the way to set the
>>> output directory to something other than target/generated-sources is
>>> to set the output configuration parameter. If this is a directory,
>>> generated files go into that directory. If that option is set in the
>>> defaults section of the schemagen config, it affects all of the
>>> generated files. Tl;dr: there is already a mechanism for changing the
>>> default output directory for all generated files.
>>>
>>> Voting to reject this patch until it's clearer why users need two
>>> mechanisms for achieving the same end.
>>
>> So, I'll remove the redundancy, but have I convinced you about the
>> problem of it not getting compiled?
> Yes, absolutely. I'm not using the plugin day-by-day (I rarely do Java
> dev these days, in fact), so direct experience by someone using the
> tool definitely takes precedence.
>
> I'm actually not sure what the right solution here is. With the
> current mechanism, the output directory can be specified on an
> vocabulary-by-vocabulary basis. Any of those locations might already
> be in the source tree already (e.g. we support the scenario where
> someone generates the .java file directly under src/main/java, so that
> the translated file can be kept under source control). So I'm not sure
> - and you are in a better position to advise on this - that just
> calling addCompileSourceRoot() on each output directory would be
> correct.




>
> Perhaps the answer is that the location of the output files should
> remain with the current configuration process, but there should be an
> additional means, via the pom.xml, to add one or more directories to
> the sources root. That could have a default value, so *if* the default
> default output is left as target/generated-sources/jena, and *if* the
> user doesn't change anything in the pom, then the output Java files
> will still compile.

To do exactly what you suggest, all we need to do is revert my change;
the build-helper-maven-plugin is already a way to add a directory to
the source roots via pom config.

However, I have a more complex idea to offer you:

Maybe redundancy is not redundancy.

We tell users: If you want to just produce sources to include in the
build, you use my new top-level property as the single source root,
and we will add it to the build, and all will be well. If you want to
do something more complex, you can use the older scheme, and it's up
to you to arrange for it to compile, perhaps using the
build-helper-maven-plugin.

To make this really painless, we'd need one more boolean property:
useOutputDirectory. Default is false, preserving old behavior. Set to
true, gets new behavior, respecting new property.

WDYT?

>
> With respect to changing the default, if writing to
> target/generated-sources simply doesn't work with Maven 3, then the
> change of default will be less intrusive, because presumably people
> are not doing that at the moment. I still think it would be worth
> raising on users@, to see if anyone would be impacted.
>
> There's enough complexity here that it's probably worth extending the
> current set of integration tests to cover this new scenario.
>

agreed:

> Ian

Re: Fixes to the schemagen plugin

[Ian Dickinson <i....@gmail.com>]

On Fri, Jun 27, 2014 at 3:31 PM, Benson Margulies <bi...@gmail.com> wrote:
> On Fri, Jun 27, 2014 at 10:05 AM, Ian Dickinson <i....@gmail.com> wrote:
>> Actually, now that I've reviewed it, there's a conflict between this
>> patch and the current behaviour. At the moment, the way to set the
>> output directory to something other than target/generated-sources is
>> to set the output configuration parameter. If this is a directory,
>> generated files go into that directory. If that option is set in the
>> defaults section of the schemagen config, it affects all of the
>> generated files. Tl;dr: there is already a mechanism for changing the
>> default output directory for all generated files.
>>
>> Voting to reject this patch until it's clearer why users need two
>> mechanisms for achieving the same end.
>
> So, I'll remove the redundancy, but have I convinced you about the
> problem of it not getting compiled?
Yes, absolutely. I'm not using the plugin day-by-day (I rarely do Java
dev these days, in fact), so direct experience by someone using the
tool definitely takes precedence.

I'm actually not sure what the right solution here is. With the
current mechanism, the output directory can be specified on an
vocabulary-by-vocabulary basis. Any of those locations might already
be in the source tree already (e.g. we support the scenario where
someone generates the .java file directly under src/main/java, so that
the translated file can be kept under source control). So I'm not sure
- and you are in a better position to advise on this - that just
calling addCompileSourceRoot() on each output directory would be
correct.

Perhaps the answer is that the location of the output files should
remain with the current configuration process, but there should be an
additional means, via the pom.xml, to add one or more directories to
the sources root. That could have a default value, so *if* the default
default output is left as target/generated-sources/jena, and *if* the
user doesn't change anything in the pom, then the output Java files
will still compile.

With respect to changing the default, if writing to
target/generated-sources simply doesn't work with Maven 3, then the
change of default will be less intrusive, because presumably people
are not doing that at the moment. I still think it would be worth
raising on users@, to see if anyone would be impacted.

There's enough complexity here that it's probably worth extending the
current set of integration tests to cover this new scenario.

Ian

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

On Fri, Jun 27, 2014 at 10:05 AM, Ian Dickinson <i....@gmail.com> wrote:
> Actually, now that I've reviewed it, there's a conflict between this
> patch and the current behaviour. At the moment, the way to set the
> output directory to something other than target/generated-sources is
> to set the output configuration parameter. If this is a directory,
> generated files go into that directory. If that option is set in the
> defaults section of the schemagen config, it affects all of the
> generated files. Tl;dr: there is already a mechanism for changing the
> default output directory for all generated files.
>
> Voting to reject this patch until it's clearer why users need two
> mechanisms for achieving the same end.

So, I'll remove the redundancy, but have I convinced you about the
problem of it not getting compiled?



>
> Ian

Re: Fixes to the schemagen plugin

[Ian Dickinson <i....@gmail.com>]

Actually, now that I've reviewed it, there's a conflict between this
patch and the current behaviour. At the moment, the way to set the
output directory to something other than target/generated-sources is
to set the output configuration parameter. If this is a directory,
generated files go into that directory. If that option is set in the
defaults section of the schemagen config, it affects all of the
generated files. Tl;dr: there is already a mechanism for changing the
default output directory for all generated files.

Voting to reject this patch until it's clearer why users need two
mechanisms for achieving the same end.

Ian

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

Once you all are comfy with the changes from last night and this
morning, I'll attempt a bit of housecleaning of that form.

On Fri, Jun 27, 2014 at 8:55 AM, Ian Dickinson <i....@gmail.com> wrote:
> On Fri, Jun 27, 2014 at 12:35 PM, Benson Margulies
> <bi...@gmail.com> wrote:
>> I wonder why the POM has apache repositories in it, since you
>> presumably have the Apache uber-parent at the top that provides them.
> Probably an artefact of the plugin being developed outside of Apache
> and then donated and integrated into the project. If you think they
> can be be removed, I'm happy with that.
>
> Ian

Re: Fixes to the schemagen plugin

[Ian Dickinson <i....@gmail.com>]

On Fri, Jun 27, 2014 at 12:35 PM, Benson Margulies
<bi...@gmail.com> wrote:
> I wonder why the POM has apache repositories in it, since you
> presumably have the Apache uber-parent at the top that provides them.
Probably an artefact of the plugin being developed outside of Apache
and then donated and integrated into the project. If you think they
can be be removed, I'm happy with that.

Ian

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

I've fixed the enforcer complaints. Generally, people building maven
plugins don't enable that check, because maven's internal
compatibility invariants make it unhelpful. But I sorted out the
versions and exclusions to make it happy.

In the version front, I assert that people want to be able to
correlate versions in JIRA with the <version/> they specify for a
plugin.

I wonder why the POM has apache repositories in it, since you
presumably have the Apache uber-parent at the top that provides them.


On Fri, Jun 27, 2014 at 6:47 AM, Andy Seaborne <an...@apache.org> wrote:
> On 27/06/14 11:12, Benson Margulies wrote:
>>
>> Shall I make the plugin have that version number, too?
>
>
> If you leave the <version> for the moment, we can discuss the long term
> policy.
>
>
>>
>> I'll fix the enforcer glitch.
>
>
> Thanks - the solution didn't jump straight out at me.
>
>         Andy
>
>
>>
>> On Fri, Jun 27, 2014 at 4:00 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> On 27/06/14 03:35, Benson Margulies wrote:
>>>>
>>>>
>>>> OK, got into JIRA, created two jiras, committed, marked them resolved.
>>>
>>>
>>>
>>> Great - there's a build problem as noted because the enforcer enforces.
>>>
>>>
>>>>
>>>> Andy, could you please add a component for the jena-maven-tools and a
>>>> version for 0.5 and 0.6 and I'll spruce them up further.
>>>
>>>
>>>
>>> Component "Maven Tool" added to JIRA.
>>>
>>> I don't what everyone else thinks but you might as well use the Jena
>>> version
>>> number.
>>>
>>> It is part of the main build/release cycle now and releases are what
>>> affect
>>> users.  It's also what people seem to use for labels on reports.
>>>
>>> TDB's (internal) version reflects disk format, but nowadays such a
>>> chanage
>>> is going to be a Jena change.  I've just removed the active JIRA version.
>>>
>>>          Andy
>>>
>>>
>>>>
>>>> On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies
>>>> <bi...@gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> Hi folks.
>>>>>
>>>>> I'm temporarily locked out of JIRA. I have some fixes to the maven
>>>>> plugin.
>>>>>
>>>>> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
>>>>> disturbed if I made a jira-less commit?
>>>>>
>>>>> The fixes are:
>>>>>
>>>>> 1: make the default output generated-sources/jena. Using
>>>>> generated-sources tangles up with 'annotations'.
>>>>>
>>>>> 2: make the output directory configurable.
>>>>>
>>>>> 3: add the output directory to the source roots.
>>>>>
>>>>> --benson
>>>
>>>
>>>
>

Re: Fixes to the schemagen plugin

[Andy Seaborne <an...@apache.org>]

On 27/06/14 11:12, Benson Margulies wrote:
> Shall I make the plugin have that version number, too?

If you leave the <version> for the moment, we can discuss the long term 
policy.

>
> I'll fix the enforcer glitch.

Thanks - the solution didn't jump straight out at me.

	Andy

>
> On Fri, Jun 27, 2014 at 4:00 AM, Andy Seaborne <an...@apache.org> wrote:
>> On 27/06/14 03:35, Benson Margulies wrote:
>>>
>>> OK, got into JIRA, created two jiras, committed, marked them resolved.
>>
>>
>> Great - there's a build problem as noted because the enforcer enforces.
>>
>>
>>>
>>> Andy, could you please add a component for the jena-maven-tools and a
>>> version for 0.5 and 0.6 and I'll spruce them up further.
>>
>>
>> Component "Maven Tool" added to JIRA.
>>
>> I don't what everyone else thinks but you might as well use the Jena version
>> number.
>>
>> It is part of the main build/release cycle now and releases are what affect
>> users.  It's also what people seem to use for labels on reports.
>>
>> TDB's (internal) version reflects disk format, but nowadays such a chanage
>> is going to be a Jena change.  I've just removed the active JIRA version.
>>
>>          Andy
>>
>>
>>>
>>> On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies <bi...@gmail.com>
>>> wrote:
>>>>
>>>> Hi folks.
>>>>
>>>> I'm temporarily locked out of JIRA. I have some fixes to the maven
>>>> plugin.
>>>>
>>>> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
>>>> disturbed if I made a jira-less commit?
>>>>
>>>> The fixes are:
>>>>
>>>> 1: make the default output generated-sources/jena. Using
>>>> generated-sources tangles up with 'annotations'.
>>>>
>>>> 2: make the output directory configurable.
>>>>
>>>> 3: add the output directory to the source roots.
>>>>
>>>> --benson
>>
>>

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

Shall I make the plugin have that version number, too?

I'll fix the enforcer glitch.

On Fri, Jun 27, 2014 at 4:00 AM, Andy Seaborne <an...@apache.org> wrote:
> On 27/06/14 03:35, Benson Margulies wrote:
>>
>> OK, got into JIRA, created two jiras, committed, marked them resolved.
>
>
> Great - there's a build problem as noted because the enforcer enforces.
>
>
>>
>> Andy, could you please add a component for the jena-maven-tools and a
>> version for 0.5 and 0.6 and I'll spruce them up further.
>
>
> Component "Maven Tool" added to JIRA.
>
> I don't what everyone else thinks but you might as well use the Jena version
> number.
>
> It is part of the main build/release cycle now and releases are what affect
> users.  It's also what people seem to use for labels on reports.
>
> TDB's (internal) version reflects disk format, but nowadays such a chanage
> is going to be a Jena change.  I've just removed the active JIRA version.
>
>         Andy
>
>
>>
>> On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies <bi...@gmail.com>
>> wrote:
>>>
>>> Hi folks.
>>>
>>> I'm temporarily locked out of JIRA. I have some fixes to the maven
>>> plugin.
>>>
>>> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
>>> disturbed if I made a jira-less commit?
>>>
>>> The fixes are:
>>>
>>> 1: make the default output generated-sources/jena. Using
>>> generated-sources tangles up with 'annotations'.
>>>
>>> 2: make the output directory configurable.
>>>
>>> 3: add the output directory to the source roots.
>>>
>>> --benson
>
>

Re: Fixes to the schemagen plugin

[Andy Seaborne <an...@apache.org>]

On 27/06/14 03:35, Benson Margulies wrote:
> OK, got into JIRA, created two jiras, committed, marked them resolved.

Great - there's a build problem as noted because the enforcer enforces.

>
> Andy, could you please add a component for the jena-maven-tools and a
> version for 0.5 and 0.6 and I'll spruce them up further.

Component "Maven Tool" added to JIRA.

I don't what everyone else thinks but you might as well use the Jena 
version number.

It is part of the main build/release cycle now and releases are what 
affect users.  It's also what people seem to use for labels on reports.

TDB's (internal) version reflects disk format, but nowadays such a 
chanage is going to be a Jena change.  I've just removed the active JIRA 
version.

	Andy

>
> On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies <bi...@gmail.com> wrote:
>> Hi folks.
>>
>> I'm temporarily locked out of JIRA. I have some fixes to the maven plugin.
>>
>> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
>> disturbed if I made a jira-less commit?
>>
>> The fixes are:
>>
>> 1: make the default output generated-sources/jena. Using
>> generated-sources tangles up with 'annotations'.
>>
>> 2: make the output directory configurable.
>>
>> 3: add the output directory to the source roots.
>>
>> --benson

Re: Fixes to the schemagen plugin

[Benson Margulies <bi...@gmail.com>]

OK, got into JIRA, created two jiras, committed, marked them resolved.

Andy, could you please add a component for the jena-maven-tools and a
version for 0.5 and 0.6 and I'll spruce them up further.

On Thu, Jun 26, 2014 at 9:42 PM, Benson Margulies <bi...@gmail.com> wrote:
> Hi folks.
>
> I'm temporarily locked out of JIRA. I have some fixes to the maven plugin.
>
> If infra unsticks me, I'll make a JIRA, but if not, would anyone be
> disturbed if I made a jira-less commit?
>
> The fixes are:
>
> 1: make the default output generated-sources/jena. Using
> generated-sources tangles up with 'annotations'.
>
> 2: make the output directory configurable.
>
> 3: add the output directory to the source roots.
>
> --benson