You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2015/01/24 19:33:31 UTC
Jena3 planning
Here is a suggestion for Jena3.
After 2.12.2 or 2.12.3,
1/ A branch for Jena2 ; master is Jena3.
2/ Switch to RDF 1.1 setting for strings
3/ Rename com.hp.hpl.jena packages to org.apache.jena
4/ Other changes
5/ Let things settle down.
6/ Release (beta? just go for it as 3.0.0?)
(3) is the disruptive step - I doubt git merge is going to be much help
after that for managing changes related to com.hp.hpl.jena packages
I wrote some migration notes for the RDF 1.1 isms and packages.
http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
There is a lot of things that could be done. I like us to avoid
over-committing ourselves. The question I have is what is *necessary*
to drive into jena 3.first.
Andy
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 22/04/15 17:45, Claude Warren wrote:
> java 8 changes the order of results returned by a hash map. I don't think
> this impacts us, though -- while they shouldn't -- some tests may depend on
> the order.
Yes, it does :-)
We have a Java8 Jenkins job that runs successfully so I don't think we
are affected.
Incorrect javadoc warnings are a complete mess though.
Andy
>
> On Wed, Apr 22, 2015 at 5:14 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 22/04/15 11:12, Claude Warren wrote:
>>
>>> +1 for the change to Jena3
>>>
>>> The timeline looks ok to me.
>>>
>>> I like the idea of holding onto jena2 for awhile in case of emergencies.
>>>
>>> As for the testing.....
>>> I think it is best to restart JENA-380 after the switch to jena3. This
>>> would effectively make JENA-380 both the conversion to Junit4 and the
>>> verification that we have all the tests we think we do . (I am assuming
>>> here that we do as Bruno suggests and each inspect the tests for the
>>> modules we are most familure with).
>>>
>>
>> Java8 looks like an interesting one. The version cycle is supposed to
>> become a regular 2y cycle IIRC.
>>
>> If necessary, to advance strict RDF 1.1, we could have 2.14.0 but I hope
>> we don't need to.
>>
>> Andy
>>
>>
>>
>>> Claude
>>>
>>> On Wed, Apr 22, 2015 at 4:30 AM, Bruno P. Kinoshita <ki...@apache.org>
>>> wrote:
>>>
>>> That would not be ideal unless you have the process scripted. I don't
>>>>> want to create rework.
>>>>>
>>>> Actually that might help. In the first commits I was still just getting
>>>> my
>>>> feet wet and understanding the current test harness, but now I already
>>>> would have to review some of my previous commits.
>>>>
>>>> I've found a few patterns that I would like to document, e.g. the
>>>> TestPackage suite found in the core module, some test suite classes being
>>>> prefixed with TS_, and easy ways to convert several classes at once by
>>>> starting by a parent class like JenaTestBase.
>>>> I'd prefer that you worked on the Jena3 branch, told me if there was any
>>>> testing or anything else that I could do to help, and then once it has
>>>> been
>>>> merged into master, I would restart the work on JENA-380.
>>>>
>>>> Side note: The only scripting that I have in place for now, is some shell
>>>> scripting (grep, find, etc) to find classes that have to be ported. But I
>>>> would like to run some other code/script in the future that could find
>>>> the
>>>> following: "Classes under the src/main/test directory, with a public
>>>> method
>>>> which name starts with test, has no parameters and doesn't contain the
>>>> @Test annotation". I missed to annotate a few JUnit3 test methods, and a
>>>> script like that could help me to review my changes before committing the
>>>> code. Is anybody aware of how to do that in a simple and quick way? I
>>>> know
>>>> that is doable with some reflection and Java code, but if there was a
>>>> simple Groovy script, sed/awk/regexes shell, or something easy to run,
>>>> that
>>>> would be very helpful too.
>>>>
>>>> Anything I can do to help?
>>>>>
>>>> Reviewing and testing my changes later would be grand!
>>>>
>>>> I'll use SonarQube and Surefire reports to count the number of tests and
>>>> the coverage before and after the changes in JENA-380. But since the
>>>> changes in JENA-380 will be orthogonal, involving several modules (in
>>>> special core, iri and arq), the more eyes we can get to review and make
>>>> sure that no tests have been disabled or broken, the merrier.
>>>> If you, Rob, Claude, and all, could inspect modules that you are familiar
>>>> with, and tell me if the tests seem to be working correctly, that would
>>>> be
>>>> of great help. How does that sound?
>>>>
>>>> ThanksBruno
>>>>
>>>> From: Andy Seaborne <an...@apache.org>
>>>> To: dev@jena.apache.org
>>>> Sent: Wednesday, April 22, 2015 10:43 AM
>>>> Subject: Re: Start Jena3
>>>>
>>>> On 21/04/15 23:14, Bruno P. Kinoshita wrote:
>>>>
>>>>> Hi Andy!
>>>>> When are you planning to start work on Jena3?
>>>>>
>>>>
>>>> Later this week but it's for discussion and flexible.
>>>>
>>>> The basic setup, the disruptive bit should be over in a few days (famous
>>>> last words!) to leave a rough, buildable setup.
>>>>
>>>> Knowing there is some in-progress changes in various places, I wanted to
>>>>>> confirm with everyone that now is good time, especially JENA-380.
>>>>>>
>>>>> I started to grok how to migrate JUnit3 tests to JUnit4, and have
>>>>>
>>>> committed some code to the JENA-380 branch. But it wouldn't be a problem
>>>> to
>>>> start from scratch after Jena3 branch has been merged into master. It
>>>> would
>>>> probably take a couple of weeks to have something that could be merged
>>>> back
>>>> into master.
>>>>
>>>> That would not be ideal unless you have the process scripted. I don't
>>>> want to create rework.
>>>>
>>>> If we postpone JENA-380, in the meantime I could:
>>>>> a) observe the commit log and get used to the new project structure
>>>>>
>>>> > b) document in the Jira issue (and/or site or Wiki) how the tests are
>>>> being migrated (e.g. remove old junit.framework.* imports when
>>>> appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
>>>> that if any test is broken later it is easier to understand what might
>>>> have happened
>>>>
>>>>> c) update the JENA-632 branch code
>>>>> d) learn more about Claude's junit-contracts project
>>>>> e) help testing parts of the code migrated?
>>>>>
>>>>> What do you think?
>>>>>
>>>>
>>>> Is there an intermediate point that can be merged into master even if
>>>> some tests are commented out/@Ignore'd temporarily?
>>>>
>>>> Anything I can do to help?
>>>>
>>>> The rename is in bulk, practical terms ::
>>>>
>>>> s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
>>>> s/import com.hp.hpl.jena/import org.apache.jena/g
>>>>
>>>> Or what about putting the upgraded tests into their own package tree
>>>> like "org.apache.jena.tests380" in master, not active so don't need to
>>>> run, just to compile, and they get updated by Eclipse renames.
>>>>
>>>>
>>>> On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
>>>> to each other, but not clashing, in the grammar.
>>>>
>>>> Andy
>>>>
>>>> Full list of branches:
>>>>
>>>> origin/JENA-380
>>>> origin/JENA-507
>>>> origin/eliminate-assignments
>>>> origin/jena-owl2
>>>> origin/streaming-update
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Bruno
>>>>>
>>>>>
>>>>> From: Andy Seaborne <an...@apache.org>
>>>>> To: dev@jena.apache.org
>>>>> Sent: Wednesday, April 22, 2015 4:28 AM
>>>>> Subject: Start Jena3
>>>>>
>>>>> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
>>>>> has happened (yet ... :-).
>>>>>
>>>>> So let's start Jena3!
>>>>>
>>>>> Summary:
>>>>> Please +1 if the process and timescale below works for you.
>>>>>
>>>>> (If I don't hear to the contrary I'll start the process later this
>>>>> week.)
>>>>>
>>>>> -------------------------------
>>>>>
>>>>> Knowing there is some in-progress changes in various places, I wanted to
>>>>> confirm with everyone that now is good time, especially JENA-380.
>>>>>
>>>>> If the timescale is inconvenient, do say so now.
>>>>> The timescale isn't driven by external needs so it's flexible.
>>>>>
>>>>>
>>>>> Proposed detailed process for the first steps.
>>>>>
>>>>> A/ Tag master with "jena-2-3-split"
>>>>>
>>>>> B/ Create remote branch jena3
>>>>> Create remote branch jena2
>>>>>
>>>>> Branch "jena3" is short-term, just to get the first steps sorted out and
>>>>> reviewed, i.e. preparation for (1). It quickly becomes "master".
>>>>>
>>>>> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>>>>>
>>>>> That is, package trees:
>>>>>
>>>>> jena-sdb/src/test/java/com/hp/hpl/jena
>>>>> jena-sdb/src/main/java/com/hp/hpl/jena
>>>>> jena-core/src/test/java/com/hp/hpl/jena
>>>>> jena-core/src/main/java/com/hp/hpl/jena
>>>>> jena-arq/src/test/java/com/hp/hpl/jena
>>>>> jena-arq/src/main/java/com/hp/hpl/jena
>>>>> jena-tdb/src/test/java/com/hp/hpl/jena
>>>>> jena-tdb/src/main/java/com/hp/hpl/jena
>>>>>
>>>>>
>>>>> I've just trialled this for the codebase and it is scarily easy to do
>>>>> with Eclipse.
>>>>>
>>>>>
>>>>> D/ Get the build working.
>>>>>
>>>>> Lots of POM updates to do but I have a build that builds.
>>>>>
>>>>> (this isn't everything - it leaves scripts, logging settings and
>>>>> resources to be done)
>>>>>
>>>>> E/ Check and review
>>>>>
>>>>> Is 24 hours enough here?
>>>>> I want to keep the window between (C) and (F) small to cope with changes
>>>>> during that window.
>>>>>
>>>>> F/ When confirmed:
>>>>>
>>>>> merge jena3 into master and push
>>>>>
>>>>> If the merge does not work, delete master, and rename jena3 to master.
>>>>>
>>>>> At this point master is jena3 and there is a jena2 branch.
>>>>>
>>>>> G/ delete jena3 branch.
>>>>>
>>>>> H/ Documentation/website
>>>>>
>>>>>
>>>>> There are quite a few choices in the details - improvements welcome.
>>>>> Experience doubly welcome. I've not done something as repo-wide as this
>>>>> before.
>>>>>
>>>>>
>>>>> Andy
>>>>>
>>>>> learnings for migration:
>>>>>
>>>>> L1/ Assembler files have class loading in them especially
>>>>> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>>>>> to org.apache.jena.tdb.
>>>>>
>>>>> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>>>>> transition to the loading step.
>>>>>
>>>>> L3/ slf4j/log4j : logger names
>>>>>
>>>>> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>>>>> jena-spatial ...)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 24/01/15 18:33, Andy Seaborne wrote:
>>>>>
>>>>>> Here is a suggestion for Jena3.
>>>>>>
>>>>>> After 2.12.2 or 2.12.3,
>>>>>>
>>>>>> 1/ A branch for Jena2 ; master is Jena3.
>>>>>> 2/ Switch to RDF 1.1 setting for strings
>>>>>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>>>>>> 4/ Other changes
>>>>>> 5/ Let things settle down.
>>>>>> 6/ Release (beta? just go for it as 3.0.0?)
>>>>>>
>>>>>> (3) is the disruptive step - I doubt git merge is going to be much help
>>>>>> after that for managing changes related to com.hp.hpl.jena packages
>>>>>>
>>>>>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>>>>>
>>>>>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>>>>>
>>>>>> There is a lot of things that could be done. I like us to avoid
>>>>>> over-committing ourselves. The question I have is what is *necessary*
>>>>>> to drive into jena 3.first.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
Re: Start Jena3
Posted by Claude Warren <cl...@xenei.com>.
java 8 changes the order of results returned by a hash map. I don't think
this impacts us, though -- while they shouldn't -- some tests may depend on
the order.
On Wed, Apr 22, 2015 at 5:14 PM, Andy Seaborne <an...@apache.org> wrote:
> On 22/04/15 11:12, Claude Warren wrote:
>
>> +1 for the change to Jena3
>>
>> The timeline looks ok to me.
>>
>> I like the idea of holding onto jena2 for awhile in case of emergencies.
>>
>> As for the testing.....
>> I think it is best to restart JENA-380 after the switch to jena3. This
>> would effectively make JENA-380 both the conversion to Junit4 and the
>> verification that we have all the tests we think we do . (I am assuming
>> here that we do as Bruno suggests and each inspect the tests for the
>> modules we are most familure with).
>>
>
> Java8 looks like an interesting one. The version cycle is supposed to
> become a regular 2y cycle IIRC.
>
> If necessary, to advance strict RDF 1.1, we could have 2.14.0 but I hope
> we don't need to.
>
> Andy
>
>
>
>> Claude
>>
>> On Wed, Apr 22, 2015 at 4:30 AM, Bruno P. Kinoshita <ki...@apache.org>
>> wrote:
>>
>> That would not be ideal unless you have the process scripted. I don't
>>>> want to create rework.
>>>>
>>> Actually that might help. In the first commits I was still just getting
>>> my
>>> feet wet and understanding the current test harness, but now I already
>>> would have to review some of my previous commits.
>>>
>>> I've found a few patterns that I would like to document, e.g. the
>>> TestPackage suite found in the core module, some test suite classes being
>>> prefixed with TS_, and easy ways to convert several classes at once by
>>> starting by a parent class like JenaTestBase.
>>> I'd prefer that you worked on the Jena3 branch, told me if there was any
>>> testing or anything else that I could do to help, and then once it has
>>> been
>>> merged into master, I would restart the work on JENA-380.
>>>
>>> Side note: The only scripting that I have in place for now, is some shell
>>> scripting (grep, find, etc) to find classes that have to be ported. But I
>>> would like to run some other code/script in the future that could find
>>> the
>>> following: "Classes under the src/main/test directory, with a public
>>> method
>>> which name starts with test, has no parameters and doesn't contain the
>>> @Test annotation". I missed to annotate a few JUnit3 test methods, and a
>>> script like that could help me to review my changes before committing the
>>> code. Is anybody aware of how to do that in a simple and quick way? I
>>> know
>>> that is doable with some reflection and Java code, but if there was a
>>> simple Groovy script, sed/awk/regexes shell, or something easy to run,
>>> that
>>> would be very helpful too.
>>>
>>> Anything I can do to help?
>>>>
>>> Reviewing and testing my changes later would be grand!
>>>
>>> I'll use SonarQube and Surefire reports to count the number of tests and
>>> the coverage before and after the changes in JENA-380. But since the
>>> changes in JENA-380 will be orthogonal, involving several modules (in
>>> special core, iri and arq), the more eyes we can get to review and make
>>> sure that no tests have been disabled or broken, the merrier.
>>> If you, Rob, Claude, and all, could inspect modules that you are familiar
>>> with, and tell me if the tests seem to be working correctly, that would
>>> be
>>> of great help. How does that sound?
>>>
>>> ThanksBruno
>>>
>>> From: Andy Seaborne <an...@apache.org>
>>> To: dev@jena.apache.org
>>> Sent: Wednesday, April 22, 2015 10:43 AM
>>> Subject: Re: Start Jena3
>>>
>>> On 21/04/15 23:14, Bruno P. Kinoshita wrote:
>>>
>>>> Hi Andy!
>>>> When are you planning to start work on Jena3?
>>>>
>>>
>>> Later this week but it's for discussion and flexible.
>>>
>>> The basic setup, the disruptive bit should be over in a few days (famous
>>> last words!) to leave a rough, buildable setup.
>>>
>>> Knowing there is some in-progress changes in various places, I wanted to
>>>>> confirm with everyone that now is good time, especially JENA-380.
>>>>>
>>>> I started to grok how to migrate JUnit3 tests to JUnit4, and have
>>>>
>>> committed some code to the JENA-380 branch. But it wouldn't be a problem
>>> to
>>> start from scratch after Jena3 branch has been merged into master. It
>>> would
>>> probably take a couple of weeks to have something that could be merged
>>> back
>>> into master.
>>>
>>> That would not be ideal unless you have the process scripted. I don't
>>> want to create rework.
>>>
>>> If we postpone JENA-380, in the meantime I could:
>>>> a) observe the commit log and get used to the new project structure
>>>>
>>> > b) document in the Jira issue (and/or site or Wiki) how the tests are
>>> being migrated (e.g. remove old junit.framework.* imports when
>>> appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
>>> that if any test is broken later it is easier to understand what might
>>> have happened
>>>
>>>> c) update the JENA-632 branch code
>>>> d) learn more about Claude's junit-contracts project
>>>> e) help testing parts of the code migrated?
>>>>
>>>> What do you think?
>>>>
>>>
>>> Is there an intermediate point that can be merged into master even if
>>> some tests are commented out/@Ignore'd temporarily?
>>>
>>> Anything I can do to help?
>>>
>>> The rename is in bulk, practical terms ::
>>>
>>> s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
>>> s/import com.hp.hpl.jena/import org.apache.jena/g
>>>
>>> Or what about putting the upgraded tests into their own package tree
>>> like "org.apache.jena.tests380" in master, not active so don't need to
>>> run, just to compile, and they get updated by Eclipse renames.
>>>
>>>
>>> On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
>>> to each other, but not clashing, in the grammar.
>>>
>>> Andy
>>>
>>> Full list of branches:
>>>
>>> origin/JENA-380
>>> origin/JENA-507
>>> origin/eliminate-assignments
>>> origin/jena-owl2
>>> origin/streaming-update
>>>
>>>
>>>
>>>
>>>
>>>> Bruno
>>>>
>>>>
>>>> From: Andy Seaborne <an...@apache.org>
>>>> To: dev@jena.apache.org
>>>> Sent: Wednesday, April 22, 2015 4:28 AM
>>>> Subject: Start Jena3
>>>>
>>>> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
>>>> has happened (yet ... :-).
>>>>
>>>> So let's start Jena3!
>>>>
>>>> Summary:
>>>> Please +1 if the process and timescale below works for you.
>>>>
>>>> (If I don't hear to the contrary I'll start the process later this
>>>> week.)
>>>>
>>>> -------------------------------
>>>>
>>>> Knowing there is some in-progress changes in various places, I wanted to
>>>> confirm with everyone that now is good time, especially JENA-380.
>>>>
>>>> If the timescale is inconvenient, do say so now.
>>>> The timescale isn't driven by external needs so it's flexible.
>>>>
>>>>
>>>> Proposed detailed process for the first steps.
>>>>
>>>> A/ Tag master with "jena-2-3-split"
>>>>
>>>> B/ Create remote branch jena3
>>>> Create remote branch jena2
>>>>
>>>> Branch "jena3" is short-term, just to get the first steps sorted out and
>>>> reviewed, i.e. preparation for (1). It quickly becomes "master".
>>>>
>>>> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>>>>
>>>> That is, package trees:
>>>>
>>>> jena-sdb/src/test/java/com/hp/hpl/jena
>>>> jena-sdb/src/main/java/com/hp/hpl/jena
>>>> jena-core/src/test/java/com/hp/hpl/jena
>>>> jena-core/src/main/java/com/hp/hpl/jena
>>>> jena-arq/src/test/java/com/hp/hpl/jena
>>>> jena-arq/src/main/java/com/hp/hpl/jena
>>>> jena-tdb/src/test/java/com/hp/hpl/jena
>>>> jena-tdb/src/main/java/com/hp/hpl/jena
>>>>
>>>>
>>>> I've just trialled this for the codebase and it is scarily easy to do
>>>> with Eclipse.
>>>>
>>>>
>>>> D/ Get the build working.
>>>>
>>>> Lots of POM updates to do but I have a build that builds.
>>>>
>>>> (this isn't everything - it leaves scripts, logging settings and
>>>> resources to be done)
>>>>
>>>> E/ Check and review
>>>>
>>>> Is 24 hours enough here?
>>>> I want to keep the window between (C) and (F) small to cope with changes
>>>> during that window.
>>>>
>>>> F/ When confirmed:
>>>>
>>>> merge jena3 into master and push
>>>>
>>>> If the merge does not work, delete master, and rename jena3 to master.
>>>>
>>>> At this point master is jena3 and there is a jena2 branch.
>>>>
>>>> G/ delete jena3 branch.
>>>>
>>>> H/ Documentation/website
>>>>
>>>>
>>>> There are quite a few choices in the details - improvements welcome.
>>>> Experience doubly welcome. I've not done something as repo-wide as this
>>>> before.
>>>>
>>>>
>>>> Andy
>>>>
>>>> learnings for migration:
>>>>
>>>> L1/ Assembler files have class loading in them especially
>>>> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>>>> to org.apache.jena.tdb.
>>>>
>>>> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>>>> transition to the loading step.
>>>>
>>>> L3/ slf4j/log4j : logger names
>>>>
>>>> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>>>> jena-spatial ...)
>>>>
>>>>
>>>>
>>>>
>>>> On 24/01/15 18:33, Andy Seaborne wrote:
>>>>
>>>>> Here is a suggestion for Jena3.
>>>>>
>>>>> After 2.12.2 or 2.12.3,
>>>>>
>>>>> 1/ A branch for Jena2 ; master is Jena3.
>>>>> 2/ Switch to RDF 1.1 setting for strings
>>>>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>>>>> 4/ Other changes
>>>>> 5/ Let things settle down.
>>>>> 6/ Release (beta? just go for it as 3.0.0?)
>>>>>
>>>>> (3) is the disruptive step - I doubt git merge is going to be much help
>>>>> after that for managing changes related to com.hp.hpl.jena packages
>>>>>
>>>>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>>>>
>>>>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>>>>
>>>>> There is a lot of things that could be done. I like us to avoid
>>>>> over-committing ourselves. The question I have is what is *necessary*
>>>>> to drive into jena 3.first.
>>>>>
>>>>> Andy
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 22/04/15 11:12, Claude Warren wrote:
> +1 for the change to Jena3
>
> The timeline looks ok to me.
>
> I like the idea of holding onto jena2 for awhile in case of emergencies.
>
> As for the testing.....
> I think it is best to restart JENA-380 after the switch to jena3. This
> would effectively make JENA-380 both the conversion to Junit4 and the
> verification that we have all the tests we think we do . (I am assuming
> here that we do as Bruno suggests and each inspect the tests for the
> modules we are most familure with).
Java8 looks like an interesting one. The version cycle is supposed to
become a regular 2y cycle IIRC.
If necessary, to advance strict RDF 1.1, we could have 2.14.0 but I hope
we don't need to.
Andy
>
> Claude
>
> On Wed, Apr 22, 2015 at 4:30 AM, Bruno P. Kinoshita <ki...@apache.org>
> wrote:
>
>>> That would not be ideal unless you have the process scripted. I don't
>>> want to create rework.
>> Actually that might help. In the first commits I was still just getting my
>> feet wet and understanding the current test harness, but now I already
>> would have to review some of my previous commits.
>>
>> I've found a few patterns that I would like to document, e.g. the
>> TestPackage suite found in the core module, some test suite classes being
>> prefixed with TS_, and easy ways to convert several classes at once by
>> starting by a parent class like JenaTestBase.
>> I'd prefer that you worked on the Jena3 branch, told me if there was any
>> testing or anything else that I could do to help, and then once it has been
>> merged into master, I would restart the work on JENA-380.
>>
>> Side note: The only scripting that I have in place for now, is some shell
>> scripting (grep, find, etc) to find classes that have to be ported. But I
>> would like to run some other code/script in the future that could find the
>> following: "Classes under the src/main/test directory, with a public method
>> which name starts with test, has no parameters and doesn't contain the
>> @Test annotation". I missed to annotate a few JUnit3 test methods, and a
>> script like that could help me to review my changes before committing the
>> code. Is anybody aware of how to do that in a simple and quick way? I know
>> that is doable with some reflection and Java code, but if there was a
>> simple Groovy script, sed/awk/regexes shell, or something easy to run, that
>> would be very helpful too.
>>
>>> Anything I can do to help?
>> Reviewing and testing my changes later would be grand!
>>
>> I'll use SonarQube and Surefire reports to count the number of tests and
>> the coverage before and after the changes in JENA-380. But since the
>> changes in JENA-380 will be orthogonal, involving several modules (in
>> special core, iri and arq), the more eyes we can get to review and make
>> sure that no tests have been disabled or broken, the merrier.
>> If you, Rob, Claude, and all, could inspect modules that you are familiar
>> with, and tell me if the tests seem to be working correctly, that would be
>> of great help. How does that sound?
>>
>> ThanksBruno
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: dev@jena.apache.org
>> Sent: Wednesday, April 22, 2015 10:43 AM
>> Subject: Re: Start Jena3
>>
>> On 21/04/15 23:14, Bruno P. Kinoshita wrote:
>>> Hi Andy!
>>> When are you planning to start work on Jena3?
>>
>> Later this week but it's for discussion and flexible.
>>
>> The basic setup, the disruptive bit should be over in a few days (famous
>> last words!) to leave a rough, buildable setup.
>>
>>>> Knowing there is some in-progress changes in various places, I wanted to
>>>> confirm with everyone that now is good time, especially JENA-380.
>>> I started to grok how to migrate JUnit3 tests to JUnit4, and have
>> committed some code to the JENA-380 branch. But it wouldn't be a problem to
>> start from scratch after Jena3 branch has been merged into master. It would
>> probably take a couple of weeks to have something that could be merged back
>> into master.
>>
>> That would not be ideal unless you have the process scripted. I don't
>> want to create rework.
>>
>>> If we postpone JENA-380, in the meantime I could:
>>> a) observe the commit log and get used to the new project structure
>> > b) document in the Jira issue (and/or site or Wiki) how the tests are
>> being migrated (e.g. remove old junit.framework.* imports when
>> appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
>> that if any test is broken later it is easier to understand what might
>> have happened
>>> c) update the JENA-632 branch code
>>> d) learn more about Claude's junit-contracts project
>>> e) help testing parts of the code migrated?
>>>
>>> What do you think?
>>
>> Is there an intermediate point that can be merged into master even if
>> some tests are commented out/@Ignore'd temporarily?
>>
>> Anything I can do to help?
>>
>> The rename is in bulk, practical terms ::
>>
>> s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
>> s/import com.hp.hpl.jena/import org.apache.jena/g
>>
>> Or what about putting the upgraded tests into their own package tree
>> like "org.apache.jena.tests380" in master, not active so don't need to
>> run, just to compile, and they get updated by Eclipse renames.
>>
>>
>> On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
>> to each other, but not clashing, in the grammar.
>>
>> Andy
>>
>> Full list of branches:
>>
>> origin/JENA-380
>> origin/JENA-507
>> origin/eliminate-assignments
>> origin/jena-owl2
>> origin/streaming-update
>>
>>
>>
>>
>>>
>>> Bruno
>>>
>>>
>>> From: Andy Seaborne <an...@apache.org>
>>> To: dev@jena.apache.org
>>> Sent: Wednesday, April 22, 2015 4:28 AM
>>> Subject: Start Jena3
>>>
>>> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
>>> has happened (yet ... :-).
>>>
>>> So let's start Jena3!
>>>
>>> Summary:
>>> Please +1 if the process and timescale below works for you.
>>>
>>> (If I don't hear to the contrary I'll start the process later this week.)
>>>
>>> -------------------------------
>>>
>>> Knowing there is some in-progress changes in various places, I wanted to
>>> confirm with everyone that now is good time, especially JENA-380.
>>>
>>> If the timescale is inconvenient, do say so now.
>>> The timescale isn't driven by external needs so it's flexible.
>>>
>>>
>>> Proposed detailed process for the first steps.
>>>
>>> A/ Tag master with "jena-2-3-split"
>>>
>>> B/ Create remote branch jena3
>>> Create remote branch jena2
>>>
>>> Branch "jena3" is short-term, just to get the first steps sorted out and
>>> reviewed, i.e. preparation for (1). It quickly becomes "master".
>>>
>>> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>>>
>>> That is, package trees:
>>>
>>> jena-sdb/src/test/java/com/hp/hpl/jena
>>> jena-sdb/src/main/java/com/hp/hpl/jena
>>> jena-core/src/test/java/com/hp/hpl/jena
>>> jena-core/src/main/java/com/hp/hpl/jena
>>> jena-arq/src/test/java/com/hp/hpl/jena
>>> jena-arq/src/main/java/com/hp/hpl/jena
>>> jena-tdb/src/test/java/com/hp/hpl/jena
>>> jena-tdb/src/main/java/com/hp/hpl/jena
>>>
>>>
>>> I've just trialled this for the codebase and it is scarily easy to do
>>> with Eclipse.
>>>
>>>
>>> D/ Get the build working.
>>>
>>> Lots of POM updates to do but I have a build that builds.
>>>
>>> (this isn't everything - it leaves scripts, logging settings and
>>> resources to be done)
>>>
>>> E/ Check and review
>>>
>>> Is 24 hours enough here?
>>> I want to keep the window between (C) and (F) small to cope with changes
>>> during that window.
>>>
>>> F/ When confirmed:
>>>
>>> merge jena3 into master and push
>>>
>>> If the merge does not work, delete master, and rename jena3 to master.
>>>
>>> At this point master is jena3 and there is a jena2 branch.
>>>
>>> G/ delete jena3 branch.
>>>
>>> H/ Documentation/website
>>>
>>>
>>> There are quite a few choices in the details - improvements welcome.
>>> Experience doubly welcome. I've not done something as repo-wide as this
>>> before.
>>>
>>>
>>> Andy
>>>
>>> learnings for migration:
>>>
>>> L1/ Assembler files have class loading in them especially
>>> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>>> to org.apache.jena.tdb.
>>>
>>> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>>> transition to the loading step.
>>>
>>> L3/ slf4j/log4j : logger names
>>>
>>> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>>> jena-spatial ...)
>>>
>>>
>>>
>>>
>>> On 24/01/15 18:33, Andy Seaborne wrote:
>>>> Here is a suggestion for Jena3.
>>>>
>>>> After 2.12.2 or 2.12.3,
>>>>
>>>> 1/ A branch for Jena2 ; master is Jena3.
>>>> 2/ Switch to RDF 1.1 setting for strings
>>>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>>>> 4/ Other changes
>>>> 5/ Let things settle down.
>>>> 6/ Release (beta? just go for it as 3.0.0?)
>>>>
>>>> (3) is the disruptive step - I doubt git merge is going to be much help
>>>> after that for managing changes related to com.hp.hpl.jena packages
>>>>
>>>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>>>
>>>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>>>
>>>> There is a lot of things that could be done. I like us to avoid
>>>> over-committing ourselves. The question I have is what is *necessary*
>>>> to drive into jena 3.first.
>>>>
>>>> Andy
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
Re: Start Jena3
Posted by Claude Warren <cl...@xenei.com>.
+1 for the change to Jena3
The timeline looks ok to me.
I like the idea of holding onto jena2 for awhile in case of emergencies.
As for the testing.....
I think it is best to restart JENA-380 after the switch to jena3. This
would effectively make JENA-380 both the conversion to Junit4 and the
verification that we have all the tests we think we do . (I am assuming
here that we do as Bruno suggests and each inspect the tests for the
modules we are most familure with).
Claude
On Wed, Apr 22, 2015 at 4:30 AM, Bruno P. Kinoshita <ki...@apache.org>
wrote:
> > That would not be ideal unless you have the process scripted. I don't
> >want to create rework.
> Actually that might help. In the first commits I was still just getting my
> feet wet and understanding the current test harness, but now I already
> would have to review some of my previous commits.
>
> I've found a few patterns that I would like to document, e.g. the
> TestPackage suite found in the core module, some test suite classes being
> prefixed with TS_, and easy ways to convert several classes at once by
> starting by a parent class like JenaTestBase.
> I'd prefer that you worked on the Jena3 branch, told me if there was any
> testing or anything else that I could do to help, and then once it has been
> merged into master, I would restart the work on JENA-380.
>
> Side note: The only scripting that I have in place for now, is some shell
> scripting (grep, find, etc) to find classes that have to be ported. But I
> would like to run some other code/script in the future that could find the
> following: "Classes under the src/main/test directory, with a public method
> which name starts with test, has no parameters and doesn't contain the
> @Test annotation". I missed to annotate a few JUnit3 test methods, and a
> script like that could help me to review my changes before committing the
> code. Is anybody aware of how to do that in a simple and quick way? I know
> that is doable with some reflection and Java code, but if there was a
> simple Groovy script, sed/awk/regexes shell, or something easy to run, that
> would be very helpful too.
>
> > Anything I can do to help?
> Reviewing and testing my changes later would be grand!
>
> I'll use SonarQube and Surefire reports to count the number of tests and
> the coverage before and after the changes in JENA-380. But since the
> changes in JENA-380 will be orthogonal, involving several modules (in
> special core, iri and arq), the more eyes we can get to review and make
> sure that no tests have been disabled or broken, the merrier.
> If you, Rob, Claude, and all, could inspect modules that you are familiar
> with, and tell me if the tests seem to be working correctly, that would be
> of great help. How does that sound?
>
> ThanksBruno
>
> From: Andy Seaborne <an...@apache.org>
> To: dev@jena.apache.org
> Sent: Wednesday, April 22, 2015 10:43 AM
> Subject: Re: Start Jena3
>
> On 21/04/15 23:14, Bruno P. Kinoshita wrote:
> > Hi Andy!
> > When are you planning to start work on Jena3?
>
> Later this week but it's for discussion and flexible.
>
> The basic setup, the disruptive bit should be over in a few days (famous
> last words!) to leave a rough, buildable setup.
>
> >> Knowing there is some in-progress changes in various places, I wanted to
> >> confirm with everyone that now is good time, especially JENA-380.
> > I started to grok how to migrate JUnit3 tests to JUnit4, and have
> committed some code to the JENA-380 branch. But it wouldn't be a problem to
> start from scratch after Jena3 branch has been merged into master. It would
> probably take a couple of weeks to have something that could be merged back
> into master.
>
> That would not be ideal unless you have the process scripted. I don't
> want to create rework.
>
> > If we postpone JENA-380, in the meantime I could:
> > a) observe the commit log and get used to the new project structure
> > b) document in the Jira issue (and/or site or Wiki) how the tests are
> being migrated (e.g. remove old junit.framework.* imports when
> appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
> that if any test is broken later it is easier to understand what might
> have happened
> > c) update the JENA-632 branch code
> > d) learn more about Claude's junit-contracts project
> > e) help testing parts of the code migrated?
> >
> > What do you think?
>
> Is there an intermediate point that can be merged into master even if
> some tests are commented out/@Ignore'd temporarily?
>
> Anything I can do to help?
>
> The rename is in bulk, practical terms ::
>
> s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
> s/import com.hp.hpl.jena/import org.apache.jena/g
>
> Or what about putting the upgraded tests into their own package tree
> like "org.apache.jena.tests380" in master, not active so don't need to
> run, just to compile, and they get updated by Eclipse renames.
>
>
> On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
> to each other, but not clashing, in the grammar.
>
> Andy
>
> Full list of branches:
>
> origin/JENA-380
> origin/JENA-507
> origin/eliminate-assignments
> origin/jena-owl2
> origin/streaming-update
>
>
>
>
> >
> > Bruno
> >
> >
> > From: Andy Seaborne <an...@apache.org>
> > To: dev@jena.apache.org
> > Sent: Wednesday, April 22, 2015 4:28 AM
> > Subject: Start Jena3
> >
> > Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
> > has happened (yet ... :-).
> >
> > So let's start Jena3!
> >
> > Summary:
> > Please +1 if the process and timescale below works for you.
> >
> > (If I don't hear to the contrary I'll start the process later this week.)
> >
> > -------------------------------
> >
> > Knowing there is some in-progress changes in various places, I wanted to
> > confirm with everyone that now is good time, especially JENA-380.
> >
> > If the timescale is inconvenient, do say so now.
> > The timescale isn't driven by external needs so it's flexible.
> >
> >
> > Proposed detailed process for the first steps.
> >
> > A/ Tag master with "jena-2-3-split"
> >
> > B/ Create remote branch jena3
> > Create remote branch jena2
> >
> > Branch "jena3" is short-term, just to get the first steps sorted out and
> > reviewed, i.e. preparation for (1). It quickly becomes "master".
> >
> > C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
> >
> > That is, package trees:
> >
> > jena-sdb/src/test/java/com/hp/hpl/jena
> > jena-sdb/src/main/java/com/hp/hpl/jena
> > jena-core/src/test/java/com/hp/hpl/jena
> > jena-core/src/main/java/com/hp/hpl/jena
> > jena-arq/src/test/java/com/hp/hpl/jena
> > jena-arq/src/main/java/com/hp/hpl/jena
> > jena-tdb/src/test/java/com/hp/hpl/jena
> > jena-tdb/src/main/java/com/hp/hpl/jena
> >
> >
> > I've just trialled this for the codebase and it is scarily easy to do
> > with Eclipse.
> >
> >
> > D/ Get the build working.
> >
> > Lots of POM updates to do but I have a build that builds.
> >
> > (this isn't everything - it leaves scripts, logging settings and
> > resources to be done)
> >
> > E/ Check and review
> >
> > Is 24 hours enough here?
> > I want to keep the window between (C) and (F) small to cope with changes
> > during that window.
> >
> > F/ When confirmed:
> >
> > merge jena3 into master and push
> >
> > If the merge does not work, delete master, and rename jena3 to master.
> >
> > At this point master is jena3 and there is a jena2 branch.
> >
> > G/ delete jena3 branch.
> >
> > H/ Documentation/website
> >
> >
> > There are quite a few choices in the details - improvements welcome.
> > Experience doubly welcome. I've not done something as repo-wide as this
> > before.
> >
> >
> > Andy
> >
> > learnings for migration:
> >
> > L1/ Assembler files have class loading in them especially
> > com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
> > to org.apache.jena.tdb.
> >
> > L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
> > transition to the loading step.
> >
> > L3/ slf4j/log4j : logger names
> >
> > L4/ Can collapse some module versions to track 3.0.0 (jena-text,
> > jena-spatial ...)
> >
> >
> >
> >
> > On 24/01/15 18:33, Andy Seaborne wrote:
> >> Here is a suggestion for Jena3.
> >>
> >> After 2.12.2 or 2.12.3,
> >>
> >> 1/ A branch for Jena2 ; master is Jena3.
> >> 2/ Switch to RDF 1.1 setting for strings
> >> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
> >> 4/ Other changes
> >> 5/ Let things settle down.
> >> 6/ Release (beta? just go for it as 3.0.0?)
> >>
> >> (3) is the disruptive step - I doubt git merge is going to be much help
> >> after that for managing changes related to com.hp.hpl.jena packages
> >>
> >> I wrote some migration notes for the RDF 1.1 isms and packages.
> >>
> >> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
> >>
> >> There is a lot of things that could be done. I like us to avoid
> >> over-committing ourselves. The question I have is what is *necessary*
> >> to drive into jena 3.first.
> >>
> >> Andy
> >
> >
> >
> >
> >
>
>
>
>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Start Jena3
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
Looking good so far Andy! I had a look at the imports and Eclipse is doing a great job!
By the way, do you have a custom coding style for Eclipse? I noticed that the Fix imports kept the space between the last character and the ";" character. If you could share your Eclipse coding style I'd start using too to ease merging of patches.
The branch isn't building yet as there are other references to the old package [1]. But after fixing some locally-only in assembler files, some tests like TestTDBAssembler start passing again, so looks like the rest of the process won't give much trouble (hopefully :)
CheersBruno
[1] https://gist.github.com/kinow/fa054b93df33bc12fb0d
From: Andy Seaborne <an...@apache.org>
To: dev@jena.apache.org
Sent: Friday, April 24, 2015 4:36 AM
Subject: Re: Start Jena3
On 22/04/15 23:26, Bruno P. Kinoshita wrote:
>> For my stuff, "Test*" are sets of tests, "TS_*" are those
>> grouped>together (e.g. by package) and "TC_*" are top level
>> collections of TS_* surefire runs all TS_, TC_ are run inside
>> Eclipse.
>
>
> So after the migration, "Test*" will be used for classes with @Test
> annotated tests. TS_ for classes annotated with @RunWith(Suite.class)
> and @SuiteClasses({$CLASSES_IN_PACKAGE}), and TC_ for classes that
> are also test suites, but may include tests in more than one
> package?
Some tests don't use "test*" when they are @Test -- Junit3 required it,
Junit4 is just style. There may be lots of "@Test test*" in ARQ because
it's converted.
The Test/TS_/TC_ convention isn't something deep; I have found it
convenient to work with sets of tests, not just run the whole lot all
the time. More so when the test suite gets a bit slow.
>
>
>> I took a very quick look - is AbstractTestPackage just not added
>> anywhere? And the constructor for AbstractModelTestBase?
>
>
> That was the last part I worked on the branch I think. I had a quick
> look at the addTest method and its Javadoc and assumed it was
> mimicking the default behavior of JUnit test suites. Looking again
> now, it clearly has some specific code for Jena, and would likely
> break some tests once I had fixed the build errors (that happened
> before in another package).
>
>
> I was also trying to make AbstractModelTestBase a simple object and
> not extend TestCase anymore. The classes with annotated @Test methods
> must have a public no arguments constructor. My idea was to add a
> setModelFactory method and try to initialize the model factory with a
> @BeforeClass or @Before. But that's still a WIP, as I haven't really
> assessed the impact of this change yet.
>
>
>> Doing it bottom up, means conversion can be done one package at a
>> time so the system is buildable - indeed, if it's bite-sized (one
>> coding session) chunks, it might as well go direct to master with
>> regular pushes.
>>
>> My picture was of an incremental approach would be to convert all
>> JUnit3 "public void test*" to JUnit4 @Test, then do the
>> TestSuites.
>
>
> I was trying to both convert TestCase to @Test annotated methods, and
> to replace the static TestSuite suite methods by classes annotated
> with @RunWith. Once Jena3 is ready, I will start migrating tests to
> JUnit4 @Test first, and then will start working on the test suites.
>
>
> Would this be the default workflow for simple JUnit3 classes with
> vanilla test suites?
> https://gist.github.com/kinow/a7c1f3af857830d4dffc
Re:
gistfile2.java
I think it's :
public static junit.framework.Test suite()
{
return new JUnit4TestAdapter(ASimpleTestCase.class) ;
}
and ASimpleTestCase need not extend JUnit3 TestCase.
It might put a extra one level node in the tests tree in Eclipse.
Caution: that's what I think it is, may not be the best way, and it's
been a while since I looked properly!
If so, then can step3 be skipped and go to pure Junit4 ASimpleTestCase
and have
public static junit.framework.Test suite()
{
TestSuite ts = new TestSuit
ts.addTest(new JUnit4TestAdapter(ASimpleTestCase.class)) ;
ts.addTest(new JUnit4TestAdapter(MoreTests.class)) ;
...
}
? Just a thought.
Andy
>
>
> Thanks! Bruno
>
>
> From: Andy Seaborne <an...@apache.org> To: dev@jena.apache.org Sent:
> Thursday, April 23, 2015 4:07 AM Subject: Re: Start Jena3
>
> On 22/04/15 04:30, Bruno P. Kinoshita wrote:
>>> That would not be ideal unless you have the process scripted. I
>>> don't want to create rework.
>> Actually that might help. In the first commits I was still just
>> getting my feet wet and understanding the current test harness,
>> but now I already would have to review some of my previous
>> commits.
>>
>> I've found a few patterns that I would like to document, e.g. the
>> TestPackage suite found in the core module, some test suite
>> classes being prefixed with TS_, and easy ways to convert several
>> classes at once by starting by a parent class like JenaTestBase.
>> I'd prefer that you worked on the Jena3 branch, told me if there
>> was any testing or anything else that I could do to help, and then
>> once it has been merged into master, I would restart the work on
>> JENA-380.
>
> For my stuff, "Test*" are sets of tests, "TS_*" are those grouped
> together (e.g. by package) and "TC_*" are top level collections of
> TS_* surefire runs all TS_, TC_ are run inside Eclipse.
>
>> Side note: The only scripting that I have in place for now, is
>> some shell scripting (grep, find, etc) to find classes that have to
>> be ported. But I would like to run some other code/script in the
>> future that could find the following: "Classes under the
>> src/main/test directory, with a public method which name starts
>> with test, has no parameters and doesn't contain the @Test
>> annotation". I missed to annotate a few JUnit3 test methods, and a
>> script like that could help me to review my changes before
>> committing the code. Is anybody aware of how to do that in a simple
>> and quick way? I know that is doable with some reflection and Java
>> code, but if there was a simple Groovy script, sed/awk/regexes
>> shell, or something easy to run, that would be very helpful too.
>>
>>> Anything I can do to help?
>> Reviewing and testing my changes later would be grand!
>
> I took a very quick look - is AbstractTestPackage just not added
> anywhere? And the constructor for AbstractModelTestBase?
>
> My picture was of an incremental approach would be to convert all
> JUnit3 "public void test*" to JUnit4 @Test, then do the TestSuites.
>
> Doing it bottom up, means conversion can be done one package at a
> time so the system is buildable - indeed, if it's bite-sized (one
> coding session) chunks, it might as well go direct to master with
> regular pushes.
>
> The bridge JUnit4TestAdapter converts classes or @Test to Junit3
> TestSuites so it'll fit in the JUnit3 framework but still work.
>
> The other bridge, @RunWith(AllTests.class), can be used to start
> with JUnit4 at the top and convert downwards.
>
> Once "JUnit4-ified", then looking at improving coverage and
> contracts can proceed in a Junit4 environment. Doing both at once
> seems like quite a big single step.
>
>
> There is some old junk manifest driven stuff around in. That'll be
> deleted in Jena3 - the packages are unused and only in jena2 for
> absolute compatibility): com.hp.hpl.jena.n3(.turtle).
>
> (( I found some unused Junit3 junk in ARQ which I'll now go and zap
> ASAP. It might be the result of (semi-)automatic conversion of the
> ARQ tests but it has left unused "extends TestSuite"
>
>
> ))
>
>> I'll use SonarQube and Surefire reports to count the number of
>> tests and the coverage before and after the changes in JENA-380.
>> But since the changes in JENA-380 will be orthogonal, involving
>> several modules (in special core, iri and arq), the more eyes we
>> can get to review and make sure that no tests have been disabled or
>> broken, the merrier. If you, Rob, Claude, and all, could inspect
>> modules that you are familiar with, and tell me if the tests seem
>> to be working correctly, that would be of great help. How does that
>> sound?
>>
>> ThanksBruno
>
> Andy
>
>
>
>
>
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 22/04/15 23:26, Bruno P. Kinoshita wrote:
>> For my stuff, "Test*" are sets of tests, "TS_*" are those
>> grouped>together (e.g. by package) and "TC_*" are top level
>> collections of TS_* surefire runs all TS_, TC_ are run inside
>> Eclipse.
>
>
> So after the migration, "Test*" will be used for classes with @Test
> annotated tests. TS_ for classes annotated with @RunWith(Suite.class)
> and @SuiteClasses({$CLASSES_IN_PACKAGE}), and TC_ for classes that
> are also test suites, but may include tests in more than one
> package?
Some tests don't use "test*" when they are @Test -- Junit3 required it,
Junit4 is just style. There may be lots of "@Test test*" in ARQ because
it's converted.
The Test/TS_/TC_ convention isn't something deep; I have found it
convenient to work with sets of tests, not just run the whole lot all
the time. More so when the test suite gets a bit slow.
>
>
>> I took a very quick look - is AbstractTestPackage just not added
>> anywhere? And the constructor for AbstractModelTestBase?
>
>
> That was the last part I worked on the branch I think. I had a quick
> look at the addTest method and its Javadoc and assumed it was
> mimicking the default behavior of JUnit test suites. Looking again
> now, it clearly has some specific code for Jena, and would likely
> break some tests once I had fixed the build errors (that happened
> before in another package).
>
>
> I was also trying to make AbstractModelTestBase a simple object and
> not extend TestCase anymore. The classes with annotated @Test methods
> must have a public no arguments constructor. My idea was to add a
> setModelFactory method and try to initialize the model factory with a
> @BeforeClass or @Before. But that's still a WIP, as I haven't really
> assessed the impact of this change yet.
>
>
>> Doing it bottom up, means conversion can be done one package at a
>> time so the system is buildable - indeed, if it's bite-sized (one
>> coding session) chunks, it might as well go direct to master with
>> regular pushes.
>>
>> My picture was of an incremental approach would be to convert all
>> JUnit3 "public void test*" to JUnit4 @Test, then do the
>> TestSuites.
>
>
> I was trying to both convert TestCase to @Test annotated methods, and
> to replace the static TestSuite suite methods by classes annotated
> with @RunWith. Once Jena3 is ready, I will start migrating tests to
> JUnit4 @Test first, and then will start working on the test suites.
>
>
> Would this be the default workflow for simple JUnit3 classes with
> vanilla test suites?
> https://gist.github.com/kinow/a7c1f3af857830d4dffc
Re:
gistfile2.java
I think it's :
public static junit.framework.Test suite()
{
return new JUnit4TestAdapter(ASimpleTestCase.class) ;
}
and ASimpleTestCase need not extend JUnit3 TestCase.
It might put a extra one level node in the tests tree in Eclipse.
Caution: that's what I think it is, may not be the best way, and it's
been a while since I looked properly!
If so, then can step3 be skipped and go to pure Junit4 ASimpleTestCase
and have
public static junit.framework.Test suite()
{
TestSuite ts = new TestSuit
ts.addTest(new JUnit4TestAdapter(ASimpleTestCase.class)) ;
ts.addTest(new JUnit4TestAdapter(MoreTests.class)) ;
...
}
? Just a thought.
Andy
>
>
> Thanks! Bruno
>
>
> From: Andy Seaborne <an...@apache.org> To: dev@jena.apache.org Sent:
> Thursday, April 23, 2015 4:07 AM Subject: Re: Start Jena3
>
> On 22/04/15 04:30, Bruno P. Kinoshita wrote:
>>> That would not be ideal unless you have the process scripted. I
>>> don't want to create rework.
>> Actually that might help. In the first commits I was still just
>> getting my feet wet and understanding the current test harness,
>> but now I already would have to review some of my previous
>> commits.
>>
>> I've found a few patterns that I would like to document, e.g. the
>> TestPackage suite found in the core module, some test suite
>> classes being prefixed with TS_, and easy ways to convert several
>> classes at once by starting by a parent class like JenaTestBase.
>> I'd prefer that you worked on the Jena3 branch, told me if there
>> was any testing or anything else that I could do to help, and then
>> once it has been merged into master, I would restart the work on
>> JENA-380.
>
> For my stuff, "Test*" are sets of tests, "TS_*" are those grouped
> together (e.g. by package) and "TC_*" are top level collections of
> TS_* surefire runs all TS_, TC_ are run inside Eclipse.
>
>> Side note: The only scripting that I have in place for now, is
>> some shell scripting (grep, find, etc) to find classes that have to
>> be ported. But I would like to run some other code/script in the
>> future that could find the following: "Classes under the
>> src/main/test directory, with a public method which name starts
>> with test, has no parameters and doesn't contain the @Test
>> annotation". I missed to annotate a few JUnit3 test methods, and a
>> script like that could help me to review my changes before
>> committing the code. Is anybody aware of how to do that in a simple
>> and quick way? I know that is doable with some reflection and Java
>> code, but if there was a simple Groovy script, sed/awk/regexes
>> shell, or something easy to run, that would be very helpful too.
>>
>>> Anything I can do to help?
>> Reviewing and testing my changes later would be grand!
>
> I took a very quick look - is AbstractTestPackage just not added
> anywhere? And the constructor for AbstractModelTestBase?
>
> My picture was of an incremental approach would be to convert all
> JUnit3 "public void test*" to JUnit4 @Test, then do the TestSuites.
>
> Doing it bottom up, means conversion can be done one package at a
> time so the system is buildable - indeed, if it's bite-sized (one
> coding session) chunks, it might as well go direct to master with
> regular pushes.
>
> The bridge JUnit4TestAdapter converts classes or @Test to Junit3
> TestSuites so it'll fit in the JUnit3 framework but still work.
>
> The other bridge, @RunWith(AllTests.class), can be used to start
> with JUnit4 at the top and convert downwards.
>
> Once "JUnit4-ified", then looking at improving coverage and
> contracts can proceed in a Junit4 environment. Doing both at once
> seems like quite a big single step.
>
>
> There is some old junk manifest driven stuff around in. That'll be
> deleted in Jena3 - the packages are unused and only in jena2 for
> absolute compatibility): com.hp.hpl.jena.n3(.turtle).
>
> (( I found some unused Junit3 junk in ARQ which I'll now go and zap
> ASAP. It might be the result of (semi-)automatic conversion of the
> ARQ tests but it has left unused "extends TestSuite"
>
>
> ))
>
>> I'll use SonarQube and Surefire reports to count the number of
>> tests and the coverage before and after the changes in JENA-380.
>> But since the changes in JENA-380 will be orthogonal, involving
>> several modules (in special core, iri and arq), the more eyes we
>> can get to review and make sure that no tests have been disabled or
>> broken, the merrier. If you, Rob, Claude, and all, could inspect
>> modules that you are familiar with, and tell me if the tests seem
>> to be working correctly, that would be of great help. How does that
>> sound?
>>
>> ThanksBruno
>
> Andy
>
>
>
>
>
Re: Start Jena3
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
>For my stuff, "Test*" are sets of tests, "TS_*" are those grouped>together (e.g. by package) and "TC_*" are top level collections of TS_*
> surefire runs all TS_, TC_ are run inside Eclipse.
So after the migration, "Test*" will be used for classes with @Test annotated tests. TS_ for classes annotated with @RunWith(Suite.class) and @SuiteClasses({$CLASSES_IN_PACKAGE}), and TC_ for classes that are also test suites, but may include tests in more than one package?
>I took a very quick look - is AbstractTestPackage just not added
>anywhere? And the constructor for AbstractModelTestBase?
That was the last part I worked on the branch I think. I had a quick look at the
addTest method and its Javadoc and assumed it was mimicking the default behavior
of JUnit test suites. Looking again now, it clearly has some specific code for
Jena, and would likely break some tests once I had fixed the build errors (that
happened before in another package).
I was also trying to make AbstractModelTestBase a simple object and not extend
TestCase anymore. The classes with annotated @Test methods must have a public
no arguments constructor. My idea was to add a setModelFactory method and try to
initialize the model factory with a @BeforeClass or @Before. But that's still a WIP,
as I haven't really assessed the impact of this change yet.
> Doing it bottom up, means conversion can be done one package at a time
>so the system is buildable - indeed, if it's bite-sized (one coding
>session) chunks, it might as well go direct to master with regular pushes.
>
>My picture was of an incremental approach would be to convert all JUnit3
> "public void test*" to JUnit4 @Test, then do the TestSuites.
I was trying to both convert TestCase to @Test annotated methods, and to replace the static TestSuite suite methods by classes annotated with @RunWith. Once Jena3 is ready, I will start migrating tests to JUnit4 @Test first, and then will start working on the test suites.
Would this be the default workflow for simple JUnit3 classes with vanilla test suites? https://gist.github.com/kinow/a7c1f3af857830d4dffc
Thanks!
Bruno
From: Andy Seaborne <an...@apache.org>
To: dev@jena.apache.org
Sent: Thursday, April 23, 2015 4:07 AM
Subject: Re: Start Jena3
On 22/04/15 04:30, Bruno P. Kinoshita wrote:
>> That would not be ideal unless you have the process scripted. I
>> don't want to create rework.
> Actually that might help. In the first commits I was still just
> getting my feet wet and understanding the current test harness, but
> now I already would have to review some of my previous commits.
>
> I've found a few patterns that I would like to document, e.g. the
> TestPackage suite found in the core module, some test suite classes
> being prefixed with TS_, and easy ways to convert several classes at
> once by starting by a parent class like JenaTestBase. I'd prefer that
> you worked on the Jena3 branch, told me if there was any testing or
> anything else that I could do to help, and then once it has been
> merged into master, I would restart the work on JENA-380.
For my stuff, "Test*" are sets of tests, "TS_*" are those grouped
together (e.g. by package) and "TC_*" are top level collections of TS_*
surefire runs all TS_, TC_ are run inside Eclipse.
> Side note: The only scripting that I have in place for now, is some
> shell scripting (grep, find, etc) to find classes that have to be
> ported. But I would like to run some other code/script in the future
> that could find the following: "Classes under the src/main/test
> directory, with a public method which name starts with test, has no
> parameters and doesn't contain the @Test annotation". I missed to
> annotate a few JUnit3 test methods, and a script like that could help
> me to review my changes before committing the code. Is anybody aware
> of how to do that in a simple and quick way? I know that is doable
> with some reflection and Java code, but if there was a simple Groovy
> script, sed/awk/regexes shell, or something easy to run, that would
> be very helpful too.
>
>> Anything I can do to help?
> Reviewing and testing my changes later would be grand!
I took a very quick look - is AbstractTestPackage just not added
anywhere? And the constructor for AbstractModelTestBase?
My picture was of an incremental approach would be to convert all JUnit3
"public void test*" to JUnit4 @Test, then do the TestSuites.
Doing it bottom up, means conversion can be done one package at a time
so the system is buildable - indeed, if it's bite-sized (one coding
session) chunks, it might as well go direct to master with regular pushes.
The bridge JUnit4TestAdapter converts classes or @Test to Junit3
TestSuites so it'll fit in the JUnit3 framework but still work.
The other bridge, @RunWith(AllTests.class), can be used to start with
JUnit4 at the top and convert downwards.
Once "JUnit4-ified", then looking at improving coverage and contracts
can proceed in a Junit4 environment. Doing both at once seems like
quite a big single step.
There is some old junk manifest driven stuff around in. That'll be
deleted in Jena3 - the packages are unused and only in jena2 for
absolute compatibility):
com.hp.hpl.jena.n3(.turtle).
((
I found some unused Junit3 junk in ARQ which I'll now go and zap ASAP.
It might be the result of (semi-)automatic conversion of the ARQ tests
but it has left unused "extends TestSuite"
))
> I'll use SonarQube and Surefire reports to count the number of tests
> and the coverage before and after the changes in JENA-380. But since
> the changes in JENA-380 will be orthogonal, involving several modules
> (in special core, iri and arq), the more eyes we can get to review
> and make sure that no tests have been disabled or broken, the
> merrier. If you, Rob, Claude, and all, could inspect modules that you
> are familiar with, and tell me if the tests seem to be working
> correctly, that would be of great help. How does that sound?
>
> ThanksBruno
Andy
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 22/04/15 04:30, Bruno P. Kinoshita wrote:
>> That would not be ideal unless you have the process scripted. I
>> don't want to create rework.
> Actually that might help. In the first commits I was still just
> getting my feet wet and understanding the current test harness, but
> now I already would have to review some of my previous commits.
>
> I've found a few patterns that I would like to document, e.g. the
> TestPackage suite found in the core module, some test suite classes
> being prefixed with TS_, and easy ways to convert several classes at
> once by starting by a parent class like JenaTestBase. I'd prefer that
> you worked on the Jena3 branch, told me if there was any testing or
> anything else that I could do to help, and then once it has been
> merged into master, I would restart the work on JENA-380.
For my stuff, "Test*" are sets of tests, "TS_*" are those grouped
together (e.g. by package) and "TC_*" are top level collections of TS_*
surefire runs all TS_, TC_ are run inside Eclipse.
> Side note: The only scripting that I have in place for now, is some
> shell scripting (grep, find, etc) to find classes that have to be
> ported. But I would like to run some other code/script in the future
> that could find the following: "Classes under the src/main/test
> directory, with a public method which name starts with test, has no
> parameters and doesn't contain the @Test annotation". I missed to
> annotate a few JUnit3 test methods, and a script like that could help
> me to review my changes before committing the code. Is anybody aware
> of how to do that in a simple and quick way? I know that is doable
> with some reflection and Java code, but if there was a simple Groovy
> script, sed/awk/regexes shell, or something easy to run, that would
> be very helpful too.
>
>> Anything I can do to help?
> Reviewing and testing my changes later would be grand!
I took a very quick look - is AbstractTestPackage just not added
anywhere? And the constructor for AbstractModelTestBase?
My picture was of an incremental approach would be to convert all JUnit3
"public void test*" to JUnit4 @Test, then do the TestSuites.
Doing it bottom up, means conversion can be done one package at a time
so the system is buildable - indeed, if it's bite-sized (one coding
session) chunks, it might as well go direct to master with regular pushes.
The bridge JUnit4TestAdapter converts classes or @Test to Junit3
TestSuites so it'll fit in the JUnit3 framework but still work.
The other bridge, @RunWith(AllTests.class), can be used to start with
JUnit4 at the top and convert downwards.
Once "JUnit4-ified", then looking at improving coverage and contracts
can proceed in a Junit4 environment. Doing both at once seems like
quite a big single step.
There is some old junk manifest driven stuff around in. That'll be
deleted in Jena3 - the packages are unused and only in jena2 for
absolute compatibility):
com.hp.hpl.jena.n3(.turtle).
((
I found some unused Junit3 junk in ARQ which I'll now go and zap ASAP.
It might be the result of (semi-)automatic conversion of the ARQ tests
but it has left unused "extends TestSuite"
))
> I'll use SonarQube and Surefire reports to count the number of tests
> and the coverage before and after the changes in JENA-380. But since
> the changes in JENA-380 will be orthogonal, involving several modules
> (in special core, iri and arq), the more eyes we can get to review
> and make sure that no tests have been disabled or broken, the
> merrier. If you, Rob, Claude, and all, could inspect modules that you
> are familiar with, and tell me if the tests seem to be working
> correctly, that would be of great help. How does that sound?
>
> ThanksBruno
Andy
Re: Start Jena3
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
> That would not be ideal unless you have the process scripted. I don't
>want to create rework.
Actually that might help. In the first commits I was still just getting my feet wet and understanding the current test harness, but now I already would have to review some of my previous commits.
I've found a few patterns that I would like to document, e.g. the TestPackage suite found in the core module, some test suite classes being prefixed with TS_, and easy ways to convert several classes at once by starting by a parent class like JenaTestBase.
I'd prefer that you worked on the Jena3 branch, told me if there was any testing or anything else that I could do to help, and then once it has been merged into master, I would restart the work on JENA-380.
Side note: The only scripting that I have in place for now, is some shell scripting (grep, find, etc) to find classes that have to be ported. But I would like to run some other code/script in the future that could find the following: "Classes under the src/main/test directory, with a public method which name starts with test, has no parameters and doesn't contain the @Test annotation". I missed to annotate a few JUnit3 test methods, and a script like that could help me to review my changes before committing the code. Is anybody aware of how to do that in a simple and quick way? I know that is doable with some reflection and Java code, but if there was a simple Groovy script, sed/awk/regexes shell, or something easy to run, that would be very helpful too.
> Anything I can do to help?
Reviewing and testing my changes later would be grand!
I'll use SonarQube and Surefire reports to count the number of tests and the coverage before and after the changes in JENA-380. But since the changes in JENA-380 will be orthogonal, involving several modules (in special core, iri and arq), the more eyes we can get to review and make sure that no tests have been disabled or broken, the merrier.
If you, Rob, Claude, and all, could inspect modules that you are familiar with, and tell me if the tests seem to be working correctly, that would be of great help. How does that sound?
ThanksBruno
From: Andy Seaborne <an...@apache.org>
To: dev@jena.apache.org
Sent: Wednesday, April 22, 2015 10:43 AM
Subject: Re: Start Jena3
On 21/04/15 23:14, Bruno P. Kinoshita wrote:
> Hi Andy!
> When are you planning to start work on Jena3?
Later this week but it's for discussion and flexible.
The basic setup, the disruptive bit should be over in a few days (famous
last words!) to leave a rough, buildable setup.
>> Knowing there is some in-progress changes in various places, I wanted to
>> confirm with everyone that now is good time, especially JENA-380.
> I started to grok how to migrate JUnit3 tests to JUnit4, and have committed some code to the JENA-380 branch. But it wouldn't be a problem to start from scratch after Jena3 branch has been merged into master. It would probably take a couple of weeks to have something that could be merged back into master.
That would not be ideal unless you have the process scripted. I don't
want to create rework.
> If we postpone JENA-380, in the meantime I could:
> a) observe the commit log and get used to the new project structure
> b) document in the Jira issue (and/or site or Wiki) how the tests are
being migrated (e.g. remove old junit.framework.* imports when
appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
that if any test is broken later it is easier to understand what might
have happened
> c) update the JENA-632 branch code
> d) learn more about Claude's junit-contracts project
> e) help testing parts of the code migrated?
>
> What do you think?
Is there an intermediate point that can be merged into master even if
some tests are commented out/@Ignore'd temporarily?
Anything I can do to help?
The rename is in bulk, practical terms ::
s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
s/import com.hp.hpl.jena/import org.apache.jena/g
Or what about putting the upgraded tests into their own package tree
like "org.apache.jena.tests380" in master, not active so don't need to
run, just to compile, and they get updated by Eclipse renames.
On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
to each other, but not clashing, in the grammar.
Andy
Full list of branches:
origin/JENA-380
origin/JENA-507
origin/eliminate-assignments
origin/jena-owl2
origin/streaming-update
>
> Bruno
>
>
> From: Andy Seaborne <an...@apache.org>
> To: dev@jena.apache.org
> Sent: Wednesday, April 22, 2015 4:28 AM
> Subject: Start Jena3
>
> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
> has happened (yet ... :-).
>
> So let's start Jena3!
>
> Summary:
> Please +1 if the process and timescale below works for you.
>
> (If I don't hear to the contrary I'll start the process later this week.)
>
> -------------------------------
>
> Knowing there is some in-progress changes in various places, I wanted to
> confirm with everyone that now is good time, especially JENA-380.
>
> If the timescale is inconvenient, do say so now.
> The timescale isn't driven by external needs so it's flexible.
>
>
> Proposed detailed process for the first steps.
>
> A/ Tag master with "jena-2-3-split"
>
> B/ Create remote branch jena3
> Create remote branch jena2
>
> Branch "jena3" is short-term, just to get the first steps sorted out and
> reviewed, i.e. preparation for (1). It quickly becomes "master".
>
> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>
> That is, package trees:
>
> jena-sdb/src/test/java/com/hp/hpl/jena
> jena-sdb/src/main/java/com/hp/hpl/jena
> jena-core/src/test/java/com/hp/hpl/jena
> jena-core/src/main/java/com/hp/hpl/jena
> jena-arq/src/test/java/com/hp/hpl/jena
> jena-arq/src/main/java/com/hp/hpl/jena
> jena-tdb/src/test/java/com/hp/hpl/jena
> jena-tdb/src/main/java/com/hp/hpl/jena
>
>
> I've just trialled this for the codebase and it is scarily easy to do
> with Eclipse.
>
>
> D/ Get the build working.
>
> Lots of POM updates to do but I have a build that builds.
>
> (this isn't everything - it leaves scripts, logging settings and
> resources to be done)
>
> E/ Check and review
>
> Is 24 hours enough here?
> I want to keep the window between (C) and (F) small to cope with changes
> during that window.
>
> F/ When confirmed:
>
> merge jena3 into master and push
>
> If the merge does not work, delete master, and rename jena3 to master.
>
> At this point master is jena3 and there is a jena2 branch.
>
> G/ delete jena3 branch.
>
> H/ Documentation/website
>
>
> There are quite a few choices in the details - improvements welcome.
> Experience doubly welcome. I've not done something as repo-wide as this
> before.
>
>
> Andy
>
> learnings for migration:
>
> L1/ Assembler files have class loading in them especially
> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
> to org.apache.jena.tdb.
>
> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
> transition to the loading step.
>
> L3/ slf4j/log4j : logger names
>
> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
> jena-spatial ...)
>
>
>
>
> On 24/01/15 18:33, Andy Seaborne wrote:
>> Here is a suggestion for Jena3.
>>
>> After 2.12.2 or 2.12.3,
>>
>> 1/ A branch for Jena2 ; master is Jena3.
>> 2/ Switch to RDF 1.1 setting for strings
>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>> 4/ Other changes
>> 5/ Let things settle down.
>> 6/ Release (beta? just go for it as 3.0.0?)
>>
>> (3) is the disruptive step - I doubt git merge is going to be much help
>> after that for managing changes related to com.hp.hpl.jena packages
>>
>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>
>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>
>> There is a lot of things that could be done. I like us to avoid
>> over-committing ourselves. The question I have is what is *necessary*
>> to drive into jena 3.first.
>>
>> Andy
>
>
>
>
>
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 21/04/15 23:14, Bruno P. Kinoshita wrote:
> Hi Andy!
> When are you planning to start work on Jena3?
Later this week but it's for discussion and flexible.
The basic setup, the disruptive bit should be over in a few days (famous
last words!) to leave a rough, buildable setup.
>> Knowing there is some in-progress changes in various places, I wanted to
>> confirm with everyone that now is good time, especially JENA-380.
> I started to grok how to migrate JUnit3 tests to JUnit4, and have committed some code to the JENA-380 branch. But it wouldn't be a problem to start from scratch after Jena3 branch has been merged into master. It would probably take a couple of weeks to have something that could be merged back into master.
That would not be ideal unless you have the process scripted. I don't
want to create rework.
> If we postpone JENA-380, in the meantime I could:
> a) observe the commit log and get used to the new project structure
> b) document in the Jira issue (and/or site or Wiki) how the tests are
being migrated (e.g. remove old junit.framework.* imports when
appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So
that if any test is broken later it is easier to understand what might
have happened
> c) update the JENA-632 branch code
> d) learn more about Claude's junit-contracts project
> e) help testing parts of the code migrated?
>
> What do you think?
Is there an intermediate point that can be merged into master even if
some tests are commented out/@Ignore'd temporarily?
Anything I can do to help?
The rename is in bulk, practical terms ::
s/package com.hp.hpl.jena/package org.apache.jena/g + mv files
s/import com.hp.hpl.jena/import org.apache.jena/g
Or what about putting the upgraded tests into their own package tree
like "org.apache.jena.tests380" in master, not active so don't need to
run, just to compile, and they get updated by Eclipse renames.
On JENA-632 - I'm aware that the GSoC JENA-491 (if approved) are close
to each other, but not clashing, in the grammar.
Andy
Full list of branches:
origin/JENA-380
origin/JENA-507
origin/eliminate-assignments
origin/jena-owl2
origin/streaming-update
>
> Bruno
>
>
> From: Andy Seaborne <an...@apache.org>
> To: dev@jena.apache.org
> Sent: Wednesday, April 22, 2015 4:28 AM
> Subject: Start Jena3
>
> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
> has happened (yet ... :-).
>
> So let's start Jena3!
>
> Summary:
> Please +1 if the process and timescale below works for you.
>
> (If I don't hear to the contrary I'll start the process later this week.)
>
> -------------------------------
>
> Knowing there is some in-progress changes in various places, I wanted to
> confirm with everyone that now is good time, especially JENA-380.
>
> If the timescale is inconvenient, do say so now.
> The timescale isn't driven by external needs so it's flexible.
>
>
> Proposed detailed process for the first steps.
>
> A/ Tag master with "jena-2-3-split"
>
> B/ Create remote branch jena3
> Create remote branch jena2
>
> Branch "jena3" is short-term, just to get the first steps sorted out and
> reviewed, i.e. preparation for (1). It quickly becomes "master".
>
> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>
> That is, package trees:
>
> jena-sdb/src/test/java/com/hp/hpl/jena
> jena-sdb/src/main/java/com/hp/hpl/jena
> jena-core/src/test/java/com/hp/hpl/jena
> jena-core/src/main/java/com/hp/hpl/jena
> jena-arq/src/test/java/com/hp/hpl/jena
> jena-arq/src/main/java/com/hp/hpl/jena
> jena-tdb/src/test/java/com/hp/hpl/jena
> jena-tdb/src/main/java/com/hp/hpl/jena
>
>
> I've just trialled this for the codebase and it is scarily easy to do
> with Eclipse.
>
>
> D/ Get the build working.
>
> Lots of POM updates to do but I have a build that builds.
>
> (this isn't everything - it leaves scripts, logging settings and
> resources to be done)
>
> E/ Check and review
>
> Is 24 hours enough here?
> I want to keep the window between (C) and (F) small to cope with changes
> during that window.
>
> F/ When confirmed:
>
> merge jena3 into master and push
>
> If the merge does not work, delete master, and rename jena3 to master.
>
> At this point master is jena3 and there is a jena2 branch.
>
> G/ delete jena3 branch.
>
> H/ Documentation/website
>
>
> There are quite a few choices in the details - improvements welcome.
> Experience doubly welcome. I've not done something as repo-wide as this
> before.
>
>
> Andy
>
> learnings for migration:
>
> L1/ Assembler files have class loading in them especially
> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
> to org.apache.jena.tdb.
>
> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
> transition to the loading step.
>
> L3/ slf4j/log4j : logger names
>
> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
> jena-spatial ...)
>
>
>
>
> On 24/01/15 18:33, Andy Seaborne wrote:
>> Here is a suggestion for Jena3.
>>
>> After 2.12.2 or 2.12.3,
>>
>> 1/ A branch for Jena2 ; master is Jena3.
>> 2/ Switch to RDF 1.1 setting for strings
>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>> 4/ Other changes
>> 5/ Let things settle down.
>> 6/ Release (beta? just go for it as 3.0.0?)
>>
>> (3) is the disruptive step - I doubt git merge is going to be much help
>> after that for managing changes related to com.hp.hpl.jena packages
>>
>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>
>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>
>> There is a lot of things that could be done. I like us to avoid
>> over-committing ourselves. The question I have is what is *necessary*
>> to drive into jena 3.first.
>>
>> Andy
>
>
>
>
>
Re: Start Jena3
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
Hi Andy!
When are you planning to start work on Jena3?
> Knowing there is some in-progress changes in various places, I wanted to
>confirm with everyone that now is good time, especially JENA-380.
I started to grok how to migrate JUnit3 tests to JUnit4, and have committed some code to the JENA-380 branch. But it wouldn't be a problem to start from scratch after Jena3 branch has been merged into master. It would probably take a couple of weeks to have something that could be merged back into master.
If we postpone JENA-380, in the meantime I could:
a) observe the commit log and get used to the new project structureb) document in the Jira issue (and/or site or Wiki) how the tests are being migrated (e.g. remove old junit.framework.* imports when appropriate, replace TestSuite's by @RunWith and @SuiteClasses, etc). So that if any test is broken later it is easier to understand what might have happened
c) update the JENA-632 branch code
d) learn more about Claude's junit-contracts project
e) help testing parts of the code migrated?
What do you think?
Bruno
From: Andy Seaborne <an...@apache.org>
To: dev@jena.apache.org
Sent: Wednesday, April 22, 2015 4:28 AM
Subject: Start Jena3
Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
has happened (yet ... :-).
So let's start Jena3!
Summary:
Please +1 if the process and timescale below works for you.
(If I don't hear to the contrary I'll start the process later this week.)
-------------------------------
Knowing there is some in-progress changes in various places, I wanted to
confirm with everyone that now is good time, especially JENA-380.
If the timescale is inconvenient, do say so now.
The timescale isn't driven by external needs so it's flexible.
Proposed detailed process for the first steps.
A/ Tag master with "jena-2-3-split"
B/ Create remote branch jena3
Create remote branch jena2
Branch "jena3" is short-term, just to get the first steps sorted out and
reviewed, i.e. preparation for (1). It quickly becomes "master".
C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
That is, package trees:
jena-sdb/src/test/java/com/hp/hpl/jena
jena-sdb/src/main/java/com/hp/hpl/jena
jena-core/src/test/java/com/hp/hpl/jena
jena-core/src/main/java/com/hp/hpl/jena
jena-arq/src/test/java/com/hp/hpl/jena
jena-arq/src/main/java/com/hp/hpl/jena
jena-tdb/src/test/java/com/hp/hpl/jena
jena-tdb/src/main/java/com/hp/hpl/jena
I've just trialled this for the codebase and it is scarily easy to do
with Eclipse.
D/ Get the build working.
Lots of POM updates to do but I have a build that builds.
(this isn't everything - it leaves scripts, logging settings and
resources to be done)
E/ Check and review
Is 24 hours enough here?
I want to keep the window between (C) and (F) small to cope with changes
during that window.
F/ When confirmed:
merge jena3 into master and push
If the merge does not work, delete master, and rename jena3 to master.
At this point master is jena3 and there is a jena2 branch.
G/ delete jena3 branch.
H/ Documentation/website
There are quite a few choices in the details - improvements welcome.
Experience doubly welcome. I've not done something as repo-wide as this
before.
Andy
learnings for migration:
L1/ Assembler files have class loading in them especially
com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
to org.apache.jena.tdb.
L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
transition to the loading step.
L3/ slf4j/log4j : logger names
L4/ Can collapse some module versions to track 3.0.0 (jena-text,
jena-spatial ...)
On 24/01/15 18:33, Andy Seaborne wrote:
> Here is a suggestion for Jena3.
>
> After 2.12.2 or 2.12.3,
>
> 1/ A branch for Jena2 ; master is Jena3.
> 2/ Switch to RDF 1.1 setting for strings
> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
> 4/ Other changes
> 5/ Let things settle down.
> 6/ Release (beta? just go for it as 3.0.0?)
>
> (3) is the disruptive step - I doubt git merge is going to be much help
> after that for managing changes related to com.hp.hpl.jena packages
>
> I wrote some migration notes for the RDF 1.1 isms and packages.
>
> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>
> There is a lot of things that could be done. I like us to avoid
> over-committing ourselves. The question I have is what is *necessary*
> to drive into jena 3.first.
>
> Andy
Re: Start Jena3
Posted by Stian Soiland-Reyes <st...@apache.org>.
You can't delete the master branch, it is protected by the
git.apache.org setup - also you can't unroll commits there. (You can
of course still do revert commits, as long as the merge was --no-ff
and thus a single commit in the log).
So perhaps wait a few days, make sure Jenkins is happy etc, before
doing the merge.
Otherwise +1 - let's go v3!
Re: Start Jena3
Posted by Stephen Allen <sa...@apache.org>.
+1 for the process and timeline
Excited for this :)
On Tue, Apr 21, 2015 at 12:28 PM, Andy Seaborne <an...@apache.org> wrote:
> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
> has happened (yet ... :-).
>
> So let's start Jena3!
>
> Summary:
> Please +1 if the process and timescale below works for you.
>
> (If I don't hear to the contrary I'll start the process later this week.)
>
> -------------------------------
>
> Knowing there is some in-progress changes in various places, I wanted to
> confirm with everyone that now is good time, especially JENA-380.
>
> If the timescale is inconvenient, do say so now.
> The timescale isn't driven by external needs so it's flexible.
>
>
> Proposed detailed process for the first steps.
>
> A/ Tag master with "jena-2-3-split"
>
> B/ Create remote branch jena3
> Create remote branch jena2
>
> Branch "jena3" is short-term, just to get the first steps sorted out and
> reviewed, i.e. preparation for (1). It quickly becomes "master".
>
> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>
> That is, package trees:
>
> jena-sdb/src/test/java/com/hp/hpl/jena
> jena-sdb/src/main/java/com/hp/hpl/jena
> jena-core/src/test/java/com/hp/hpl/jena
> jena-core/src/main/java/com/hp/hpl/jena
> jena-arq/src/test/java/com/hp/hpl/jena
> jena-arq/src/main/java/com/hp/hpl/jena
> jena-tdb/src/test/java/com/hp/hpl/jena
> jena-tdb/src/main/java/com/hp/hpl/jena
>
>
> I've just trialled this for the codebase and it is scarily easy to do with
> Eclipse.
>
>
> D/ Get the build working.
>
> Lots of POM updates to do but I have a build that builds.
>
> (this isn't everything - it leaves scripts, logging settings and resources
> to be done)
>
> E/ Check and review
>
> Is 24 hours enough here?
> I want to keep the window between (C) and (F) small to cope with changes
> during that window.
>
> F/ When confirmed:
>
> merge jena3 into master and push
>
> If the merge does not work, delete master, and rename jena3 to master.
>
> At this point master is jena3 and there is a jena2 branch.
>
> G/ delete jena3 branch.
>
> H/ Documentation/website
>
>
> There are quite a few choices in the details - improvements welcome.
> Experience doubly welcome. I've not done something as repo-wide as this
> before.
>
>
> Andy
>
> learnings for migration:
>
> L1/ Assembler files have class loading in them especially
> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing to
> org.apache.jena.tdb.
>
> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
> transition to the loading step.
>
> L3/ slf4j/log4j : logger names
>
> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
> jena-spatial ...)
>
>
> On 24/01/15 18:33, Andy Seaborne wrote:
>
>> Here is a suggestion for Jena3.
>>
>> After 2.12.2 or 2.12.3,
>>
>> 1/ A branch for Jena2 ; master is Jena3.
>> 2/ Switch to RDF 1.1 setting for strings
>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>> 4/ Other changes
>> 5/ Let things settle down.
>> 6/ Release (beta? just go for it as 3.0.0?)
>>
>> (3) is the disruptive step - I doubt git merge is going to be much help
>> after that for managing changes related to com.hp.hpl.jena packages
>>
>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>
>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>
>> There is a lot of things that could be done. I like us to avoid
>> over-committing ourselves. The question I have is what is *necessary*
>> to drive into jena 3.first.
>>
>> Andy
>>
>
>
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
>>> At this point master is jena3 and there is a jena2 branch.
>>
>> What is the purpose of the Jena2 branch, are we intended to provide
>> ongoing maintenance and back porting of 3.x fixes for the 2.x branch? Or
>> is it simply a convenient place for those who want to stick to Jena2 to
>> work from?
>
> The jena2 is there to leave choices as long as possible.
>
> 1/ backing out jena3.
>
> 2/ (maybe) doing a 2.13.1 in parallel to 3.0.0
>
> 3/ We can put in a jena2 build in Jenkins.
4/ java7 / jena2
java8 / jena3
(The transaction and COW-B+Trees are java8 which reminded me)
Andy
Java7(oracle) - End of Public Updates - April 2015 (or revised to be later)
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
On 21/04/15 18:14, Rob Vesse wrote:
> Sounds like a plan, I don't have any strong preference on time scales
>
>> I've just trialled this for the codebase and it is scarily easy to do
>> with Eclipse.
>
> Yep tis nice isn't it, I presume you have a version of Eclipse with proper
> Git integration or you do the appropriate Git invocations so Git registers
> these as file renames in so far as is possible?
I was using Luna 4.4.1. I'll upgrade to 4.4.2 before I do this for
real. I was going to do it from scratch a second time. The package
rename is quite quick; I'd liek to take a more systematic approach to
the POMs.
>> F/ When confirmed:
>>
>> merge jena3 into master and push
>>
>> If the merge does not work, delete master, and rename jena3 to master.
>
> Provided you start by branching from master I see no reason why merging
> would not work
>
> What might be slightly better would be the following:
>
> 1/ Announce that master is frozen on the mailing list, wait a little while
> to make sure everyone sees
> 2/ Merge master into jena3 to grab any small changes that have happened
> since the split
> 3/ Fix up any conflicts and build failures introduced as a result
> 4/ Merge jena3 back into master
Good suggestion.
>
>>
>> At this point master is jena3 and there is a jena2 branch.
>
> What is the purpose of the Jena2 branch, are we intended to provide
> ongoing maintenance and back porting of 3.x fixes for the 2.x branch? Or
> is it simply a convenient place for those who want to stick to Jena2 to
> work from?
The jena2 is there to leave choices as long as possible.
1/ backing out jena3.
2/ (maybe) doing a 2.13.1 in parallel to 3.0.0
3/ We can put in a jena2 build in Jenkins.
> If we as a project don't intend to maintain the 2.x line then I see no
> reason to have a Jena2 branch and having the jena-2-3-split tag should be
> sufficient for those who want to continue to stick to 2.x
I don't speak for every one but, personally, it's impractical to run two
lines for any length of time.
> Historically we have never supported multiple versions and our advice has
> always been upgrade when a user complains of a bug in a past version so I
> don't see that we would want to start now
I'm being cautious :-)
>
>>
>> G/ delete jena3 branch.
>>
>> H/ Documentation/website
>>
>>
>> There are quite a few choices in the details - improvements welcome.
>> Experience doubly welcome. I've not done something as repo-wide as this
>> before.
>>
>>
>> Andy
>>
>> learnings for migration:
>>
>> L1/ Assembler files have class loading in them especially
>> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>> to org.apache.jena.tdb.
>
> Agreed, perhaps building a general purpose package redirection mechanism
> into the Assembler machinery would be a good idea
>
>>
>> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>> transition to the loading step.
>
> Yep that should be easy to do and in this case the redirection mechanism
> already exists I believe?
>
>>
>> L3/ slf4j/log4j : logger names
>
> The logger names or the package names mentioned in the log configurations?
log configurations.
(Something for the migration doc)
I was assuming if the logger is a class, and the class is repackaged,
the logger changes.
>>
>> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>> jena-spatial ...)
>
> Yep that would be nice and probably required since changing the package
> names is certainly a breaking change for most people and setting
> everything to be 3.0.0 would make it clear a major bump happens
>
> I would go so far as to suggest that we bump all versions (bar parent) to
> a 3.0.0 baseline just so no-one can moan that we didn't follow SemVer
> practises properly
jena-text, jena-spatial, jena-sdb, jena-iri, jena-csv
(style-copy relics of separate releases)
elephas and jdbc as well?
I'll think about TDB v2 or v3 (probably v3) - the version is semi-tied
to the disk format.
(I have fully scalable transactions working that is an upwards
compatible format change, but not reversible. Basically, a
root-and-branch rewrite of transactions with the possibility of hooks
into other systems (separate transaction controller; distribution) +
copy-on-write trees. Too much for the Jena3 timeframe.)
Andy
>
> Rob
>
>>
>>
>> On 24/01/15 18:33, Andy Seaborne wrote:
>>> Here is a suggestion for Jena3.
>>>
>>> After 2.12.2 or 2.12.3,
>>>
>>> 1/ A branch for Jena2 ; master is Jena3.
>>> 2/ Switch to RDF 1.1 setting for strings
>>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>>> 4/ Other changes
>>> 5/ Let things settle down.
>>> 6/ Release (beta? just go for it as 3.0.0?)
>>>
>>> (3) is the disruptive step - I doubt git merge is going to be much help
>>> after that for managing changes related to com.hp.hpl.jena packages
>>>
>>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>>
>>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>>
>>> There is a lot of things that could be done. I like us to avoid
>>> over-committing ourselves. The question I have is what is *necessary*
>>> to drive into jena 3.first.
>>>
>>> Andy
>>
>
>
>
>
Re: Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
Starting ...
[branch "jena2"]
remote = origin
merge = refs/heads/jena2
[branch "jena3"]
remote = origin
merge = refs/heads/jena3
On 21/04/15 18:14, Rob Vesse wrote:
> Sounds like a plan, I don't have any strong preference on time scales
>
> Other comments inline:
>
> On 21/04/2015 09:28, "Andy Seaborne" <an...@apache.org> wrote:
>
>> Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
>> has happened (yet ... :-).
>>
>> So let's start Jena3!
>>
>> Summary:
>> Please +1 if the process and timescale below works for you.
>>
>> (If I don't hear to the contrary I'll start the process later this week.)
>>
>> -------------------------------
>>
>> Knowing there is some in-progress changes in various places, I wanted to
>> confirm with everyone that now is good time, especially JENA-380.
>>
>> If the timescale is inconvenient, do say so now.
>> The timescale isn't driven by external needs so it's flexible.
>>
>>
>> Proposed detailed process for the first steps.
>>
>> A/ Tag master with "jena-2-3-split"
>>
>> B/ Create remote branch jena3
>> Create remote branch jena2
>>
>> Branch "jena3" is short-term, just to get the first steps sorted out and
>> reviewed, i.e. preparation for (1). It quickly becomes "master".
>>
>> C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>>
>> That is, package trees:
>>
>> jena-sdb/src/test/java/com/hp/hpl/jena
>> jena-sdb/src/main/java/com/hp/hpl/jena
>> jena-core/src/test/java/com/hp/hpl/jena
>> jena-core/src/main/java/com/hp/hpl/jena
>> jena-arq/src/test/java/com/hp/hpl/jena
>> jena-arq/src/main/java/com/hp/hpl/jena
>> jena-tdb/src/test/java/com/hp/hpl/jena
>> jena-tdb/src/main/java/com/hp/hpl/jena
>>
>>
>> I've just trialled this for the codebase and it is scarily easy to do
>> with Eclipse.
>
> Yep tis nice isn't it, I presume you have a version of Eclipse with proper
> Git integration or you do the appropriate Git invocations so Git registers
> these as file renames in so far as is possible?
>
>>
>>
>> D/ Get the build working.
>>
>> Lots of POM updates to do but I have a build that builds.
>>
>> (this isn't everything - it leaves scripts, logging settings and
>> resources to be done)
>>
>> E/ Check and review
>>
>> Is 24 hours enough here?
>> I want to keep the window between (C) and (F) small to cope with changes
>> during that window.
>>
>> F/ When confirmed:
>>
>> merge jena3 into master and push
>>
>> If the merge does not work, delete master, and rename jena3 to master.
>
> Provided you start by branching from master I see no reason why merging
> would not work
>
> What might be slightly better would be the following:
>
> 1/ Announce that master is frozen on the mailing list, wait a little while
> to make sure everyone sees
> 2/ Merge master into jena3 to grab any small changes that have happened
> since the split
> 3/ Fix up any conflicts and build failures introduced as a result
> 4/ Merge jena3 back into master
>
>>
>> At this point master is jena3 and there is a jena2 branch.
>
> What is the purpose of the Jena2 branch, are we intended to provide
> ongoing maintenance and back porting of 3.x fixes for the 2.x branch? Or
> is it simply a convenient place for those who want to stick to Jena2 to
> work from?
>
> If we as a project don't intend to maintain the 2.x line then I see no
> reason to have a Jena2 branch and having the jena-2-3-split tag should be
> sufficient for those who want to continue to stick to 2.x
>
> Historically we have never supported multiple versions and our advice has
> always been upgrade when a user complains of a bug in a past version so I
> don't see that we would want to start now
>
>>
>> G/ delete jena3 branch.
>>
>> H/ Documentation/website
>>
>>
>> There are quite a few choices in the details - improvements welcome.
>> Experience doubly welcome. I've not done something as repo-wide as this
>> before.
>>
>>
>> Andy
>>
>> learnings for migration:
>>
>> L1/ Assembler files have class loading in them especially
>> com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>> to org.apache.jena.tdb.
>
> Agreed, perhaps building a general purpose package redirection mechanism
> into the Assembler machinery would be a good idea
>
>>
>> L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>> transition to the loading step.
>
> Yep that should be easy to do and in this case the redirection mechanism
> already exists I believe?
>
>>
>> L3/ slf4j/log4j : logger names
>
> The logger names or the package names mentioned in the log configurations?
>
>>
>> L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>> jena-spatial ...)
>
> Yep that would be nice and probably required since changing the package
> names is certainly a breaking change for most people and setting
> everything to be 3.0.0 would make it clear a major bump happens
>
> I would go so far as to suggest that we bump all versions (bar parent) to
> a 3.0.0 baseline just so no-one can moan that we didn't follow SemVer
> practises properly
>
> Rob
>
>>
>>
>> On 24/01/15 18:33, Andy Seaborne wrote:
>>> Here is a suggestion for Jena3.
>>>
>>> After 2.12.2 or 2.12.3,
>>>
>>> 1/ A branch for Jena2 ; master is Jena3.
>>> 2/ Switch to RDF 1.1 setting for strings
>>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>>> 4/ Other changes
>>> 5/ Let things settle down.
>>> 6/ Release (beta? just go for it as 3.0.0?)
>>>
>>> (3) is the disruptive step - I doubt git merge is going to be much help
>>> after that for managing changes related to com.hp.hpl.jena packages
>>>
>>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>>
>>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>>
>>> There is a lot of things that could be done. I like us to avoid
>>> over-committing ourselves. The question I have is what is *necessary*
>>> to drive into jena 3.first.
>>>
>>> Andy
>>
>
>
>
>
Re: Start Jena3
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Sounds like a plan, I don't have any strong preference on time scales
Other comments inline:
On 21/04/2015 09:28, "Andy Seaborne" <an...@apache.org> wrote:
>Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
>has happened (yet ... :-).
>
>So let's start Jena3!
>
>Summary:
>Please +1 if the process and timescale below works for you.
>
>(If I don't hear to the contrary I'll start the process later this week.)
>
>-------------------------------
>
>Knowing there is some in-progress changes in various places, I wanted to
>confirm with everyone that now is good time, especially JENA-380.
>
>If the timescale is inconvenient, do say so now.
>The timescale isn't driven by external needs so it's flexible.
>
>
>Proposed detailed process for the first steps.
>
>A/ Tag master with "jena-2-3-split"
>
>B/ Create remote branch jena3
> Create remote branch jena2
>
>Branch "jena3" is short-term, just to get the first steps sorted out and
>reviewed, i.e. preparation for (1). It quickly becomes "master".
>
>C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
>
>That is, package trees:
>
>jena-sdb/src/test/java/com/hp/hpl/jena
>jena-sdb/src/main/java/com/hp/hpl/jena
>jena-core/src/test/java/com/hp/hpl/jena
>jena-core/src/main/java/com/hp/hpl/jena
>jena-arq/src/test/java/com/hp/hpl/jena
>jena-arq/src/main/java/com/hp/hpl/jena
>jena-tdb/src/test/java/com/hp/hpl/jena
>jena-tdb/src/main/java/com/hp/hpl/jena
>
>
>I've just trialled this for the codebase and it is scarily easy to do
>with Eclipse.
Yep tis nice isn't it, I presume you have a version of Eclipse with proper
Git integration or you do the appropriate Git invocations so Git registers
these as file renames in so far as is possible?
>
>
>D/ Get the build working.
>
>Lots of POM updates to do but I have a build that builds.
>
>(this isn't everything - it leaves scripts, logging settings and
>resources to be done)
>
>E/ Check and review
>
>Is 24 hours enough here?
>I want to keep the window between (C) and (F) small to cope with changes
>during that window.
>
>F/ When confirmed:
>
> merge jena3 into master and push
>
>If the merge does not work, delete master, and rename jena3 to master.
Provided you start by branching from master I see no reason why merging
would not work
What might be slightly better would be the following:
1/ Announce that master is frozen on the mailing list, wait a little while
to make sure everyone sees
2/ Merge master into jena3 to grab any small changes that have happened
since the split
3/ Fix up any conflicts and build failures introduced as a result
4/ Merge jena3 back into master
>
>At this point master is jena3 and there is a jena2 branch.
What is the purpose of the Jena2 branch, are we intended to provide
ongoing maintenance and back porting of 3.x fixes for the 2.x branch? Or
is it simply a convenient place for those who want to stick to Jena2 to
work from?
If we as a project don't intend to maintain the 2.x line then I see no
reason to have a Jena2 branch and having the jena-2-3-split tag should be
sufficient for those who want to continue to stick to 2.x
Historically we have never supported multiple versions and our advice has
always been upgrade when a user complains of a bug in a past version so I
don't see that we would want to start now
>
>G/ delete jena3 branch.
>
>H/ Documentation/website
>
>
>There are quite a few choices in the details - improvements welcome.
>Experience doubly welcome. I've not done something as repo-wide as this
>before.
>
>
> Andy
>
>learnings for migration:
>
>L1/ Assembler files have class loading in them especially
>com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
>to org.apache.jena.tdb.
Agreed, perhaps building a general purpose package redirection mechanism
into the Assembler machinery would be a good idea
>
>L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
>transition to the loading step.
Yep that should be easy to do and in this case the redirection mechanism
already exists I believe?
>
>L3/ slf4j/log4j : logger names
The logger names or the package names mentioned in the log configurations?
>
>L4/ Can collapse some module versions to track 3.0.0 (jena-text,
>jena-spatial ...)
Yep that would be nice and probably required since changing the package
names is certainly a breaking change for most people and setting
everything to be 3.0.0 would make it clear a major bump happens
I would go so far as to suggest that we bump all versions (bar parent) to
a 3.0.0 baseline just so no-one can moan that we didn't follow SemVer
practises properly
Rob
>
>
>On 24/01/15 18:33, Andy Seaborne wrote:
>> Here is a suggestion for Jena3.
>>
>> After 2.12.2 or 2.12.3,
>>
>> 1/ A branch for Jena2 ; master is Jena3.
>> 2/ Switch to RDF 1.1 setting for strings
>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>> 4/ Other changes
>> 5/ Let things settle down.
>> 6/ Release (beta? just go for it as 3.0.0?)
>>
>> (3) is the disruptive step - I doubt git merge is going to be much help
>> after that for managing changes related to com.hp.hpl.jena packages
>>
>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>
>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>
>> There is a lot of things that could be done. I like us to avoid
>> over-committing ourselves. The question I have is what is *necessary*
>> to drive into jena 3.first.
>>
>> Andy
>
Start Jena3
Posted by Andy Seaborne <an...@apache.org>.
Jena 2.13.0 has been out for out for a few weeks now and nothing too bad
has happened (yet ... :-).
So let's start Jena3!
Summary:
Please +1 if the process and timescale below works for you.
(If I don't hear to the contrary I'll start the process later this week.)
-------------------------------
Knowing there is some in-progress changes in various places, I wanted to
confirm with everyone that now is good time, especially JENA-380.
If the timescale is inconvenient, do say so now.
The timescale isn't driven by external needs so it's flexible.
Proposed detailed process for the first steps.
A/ Tag master with "jena-2-3-split"
B/ Create remote branch jena3
Create remote branch jena2
Branch "jena3" is short-term, just to get the first steps sorted out and
reviewed, i.e. preparation for (1). It quickly becomes "master".
C/ Do step 3 : Rename com.hp.hpl.jena packages to org.apache.jena
That is, package trees:
jena-sdb/src/test/java/com/hp/hpl/jena
jena-sdb/src/main/java/com/hp/hpl/jena
jena-core/src/test/java/com/hp/hpl/jena
jena-core/src/main/java/com/hp/hpl/jena
jena-arq/src/test/java/com/hp/hpl/jena
jena-arq/src/main/java/com/hp/hpl/jena
jena-tdb/src/test/java/com/hp/hpl/jena
jena-tdb/src/main/java/com/hp/hpl/jena
I've just trialled this for the codebase and it is scarily easy to do
with Eclipse.
D/ Get the build working.
Lots of POM updates to do but I have a build that builds.
(this isn't everything - it leaves scripts, logging settings and
resources to be done)
E/ Check and review
Is 24 hours enough here?
I want to keep the window between (C) and (F) small to cope with changes
during that window.
F/ When confirmed:
merge jena3 into master and push
If the merge does not work, delete master, and rename jena3 to master.
At this point master is jena3 and there is a jena2 branch.
G/ delete jena3 branch.
H/ Documentation/website
There are quite a few choices in the details - improvements welcome.
Experience doubly welcome. I've not done something as repo-wide as this
before.
Andy
learnings for migration:
L1/ Assembler files have class loading in them especially
com.hpl.hpl.jena.tdb. Maybe worth trapping during loading and changing
to org.apache.jena.tdb.
L2/ "java:" custom filter functions (ARQ and Leviathan). Again, add
transition to the loading step.
L3/ slf4j/log4j : logger names
L4/ Can collapse some module versions to track 3.0.0 (jena-text,
jena-spatial ...)
On 24/01/15 18:33, Andy Seaborne wrote:
> Here is a suggestion for Jena3.
>
> After 2.12.2 or 2.12.3,
>
> 1/ A branch for Jena2 ; master is Jena3.
> 2/ Switch to RDF 1.1 setting for strings
> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
> 4/ Other changes
> 5/ Let things settle down.
> 6/ Release (beta? just go for it as 3.0.0?)
>
> (3) is the disruptive step - I doubt git merge is going to be much help
> after that for managing changes related to com.hp.hpl.jena packages
>
> I wrote some migration notes for the RDF 1.1 isms and packages.
>
> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>
> There is a lot of things that could be done. I like us to avoid
> over-committing ourselves. The question I have is what is *necessary*
> to drive into jena 3.first.
>
> Andy
Re: Jena3 planning [SDB]
Posted by Andy Seaborne <an...@apache.org>.
On 27/01/15 07:10, Claude Warren wrote:
> I still think we need SDB or an SDB like package. My reasoning is that
> there are a number of places (think cloud providers, think large
> corporations that have internal data centers) where you can get a standard
> DB with all the backup and restore capability built in. The ability to
> deploy on an existing well known infrastructure makes Jena an easier sell
> to management.
All true but.
SDB has been moved to RDF 1.1 because the code changes weren't too big.
(Data reload will be needed.) But we do not test SDB on each "supported"
database; it would need hardware, software management, licenses and
people-time.
We're growing - support growth is on answers.semanticweb and
stackoverflow where some great people give really thoughtful answers; it
is a great skill.
And at the same time, it feels to me that users@ questions are getting
harder :-) There is more benefit for user community for my time to go on
TDB and Fuseki.
The reality is that there are 3 ways to work with open source:
contribute to it, resource it, hope stuff happens. If cloud
providers/large corp want it to generate revenue out of it, well, there
are ways to do that other than "hope".
Andy
Re: Jena3 planning
Posted by Claude Warren <cl...@xenei.com>.
I still think we need SDB or an SDB like package. My reasoning is that
there are a number of places (think cloud providers, think large
corporations that have internal data centers) where you can get a standard
DB with all the backup and restore capability built in. The ability to
deploy on an existing well known infrastructure makes Jena an easier sell
to management.
On Mon, Jan 26, 2015 at 9:56 PM, Stian Soiland-Reyes <st...@apache.org>
wrote:
> +1 for the plan.
>
> Some thoughts for "other changes" I have wanted to get my teeth into:
>
> a) remove legacy Model reader/writer API
> b) move RIOT into jena-core (as was suggested), keep SPARQL in arq
> b2) rename jena-arq to jena-query?
> b3) rename jena-tdb to jena-triplestore ?
> b4) retire jena-sdb? Some interest still..
>
> c) cleanup of RDFDataMgr - do we need all those methods?
> d) use java.nio.file.Path instead of java.util.File all over
> e) get rid of any remaining Class.forName and add registries/ServiceLoader
> patterns as needed
>
> e) and b) can be done already in 2.x.
>
> To do git merge across file renamed requires some hand holding, but seems
> to work surprisingly well. New files on the branch will appear in the wrong
> place, and any new import statements in existing files will be wrong on the
> other branch.
>
> Beta or 3.0.0? Well, that depends on what goes into the other changes! ;-)
> On 24 Jan 2015 18:34, "Andy Seaborne" <an...@apache.org> wrote:
>
> > Here is a suggestion for Jena3.
> >
> > After 2.12.2 or 2.12.3,
> >
> > 1/ A branch for Jena2 ; master is Jena3.
> > 2/ Switch to RDF 1.1 setting for strings
> > 3/ Rename com.hp.hpl.jena packages to org.apache.jena
> > 4/ Other changes
> > 5/ Let things settle down.
> > 6/ Release (beta? just go for it as 3.0.0?)
> >
> > (3) is the disruptive step - I doubt git merge is going to be much help
> > after that for managing changes related to com.hp.hpl.jena packages
> >
> > I wrote some migration notes for the RDF 1.1 isms and packages.
> >
> > http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
> >
> > There is a lot of things that could be done. I like us to avoid
> > over-committing ourselves. The question I have is what is *necessary* to
> > drive into jena 3.first.
> >
> > Andy
> >
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Jena3 planning
Posted by Stian Soiland-Reyes <st...@apache.org>.
+1 for the plan.
Some thoughts for "other changes" I have wanted to get my teeth into:
a) remove legacy Model reader/writer API
b) move RIOT into jena-core (as was suggested), keep SPARQL in arq
b2) rename jena-arq to jena-query?
b3) rename jena-tdb to jena-triplestore ?
b4) retire jena-sdb? Some interest still..
c) cleanup of RDFDataMgr - do we need all those methods?
d) use java.nio.file.Path instead of java.util.File all over
e) get rid of any remaining Class.forName and add registries/ServiceLoader
patterns as needed
e) and b) can be done already in 2.x.
To do git merge across file renamed requires some hand holding, but seems
to work surprisingly well. New files on the branch will appear in the wrong
place, and any new import statements in existing files will be wrong on the
other branch.
Beta or 3.0.0? Well, that depends on what goes into the other changes! ;-)
On 24 Jan 2015 18:34, "Andy Seaborne" <an...@apache.org> wrote:
> Here is a suggestion for Jena3.
>
> After 2.12.2 or 2.12.3,
>
> 1/ A branch for Jena2 ; master is Jena3.
> 2/ Switch to RDF 1.1 setting for strings
> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
> 4/ Other changes
> 5/ Let things settle down.
> 6/ Release (beta? just go for it as 3.0.0?)
>
> (3) is the disruptive step - I doubt git merge is going to be much help
> after that for managing changes related to com.hp.hpl.jena packages
>
> I wrote some migration notes for the RDF 1.1 isms and packages.
>
> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>
> There is a lot of things that could be done. I like us to avoid
> over-committing ourselves. The question I have is what is *necessary* to
> drive into jena 3.first.
>
> Andy
>
Re: Jena3 planning
Posted by Andy Seaborne <an...@apache.org>.
On 26/01/15 19:18, Rob Vesse wrote:
> git merge is generally not too terrible at following file renames provided
> you do them in a single git commit so it recognises that something was
> renamed. However the package name changes (and thus the import statement
> changes) may mean that cherry picking or merging across branches requires
> a certain amount of manual fix up.
>
> I guess the better question is "What is our support policy for Jena 2?"
>
> I assume that there would need to be a transitional period during which
> bug fixes go to both Jena 2 and 3 but in the longer term the intent would
> be to get everyone to migrate to Jena 3 and then stop supporting Jena 2.
> Since it would still exist as a branch people could choose to maintain it
> themselves once the project dropped support if they really couldn't
> migrate to Jena 2.
Yes - you have hit big issue about contributions.
On the migration:
The more changes that Jena3 makes, the slower the migration from Jena2.
So, for example, changing the API (model.read) without going via a
deprecated/remove sequence over several versions, is then a barrier to
people moving over ... and longer term Jena2 support implications.
> If Jena 3 is primarily the package rename and RDF 1.1 on by default then I
> don't see any reason to use a Beta modifier. The essential code remains
> the same and is well tested and deployed so the Beta moniker seems
> inappropriate.
Yes - that's a good framework for deciding on changes. Things that need
to be tried out, and hence suggest a "beta", should go out on a longer
cycle.
> I read your migration instructions and think they are pretty clear and
> concise. I hope that people won't have too many issues migrating forwards
> to Jena 3 (famous last words!). I suspect the biggest problem will be
> exactly the same as we see now, old code and tools out there referencing
> old versions of the libraries and creating class path conflicts when users
> (inadvertently or otherwise) end up mixing different library versions.
So - no change there then :-)
> I think ultimately our long term strategy has to be "Please move to Jena
> 3" because we won't have the bandwidth to maintain two parallel branches
> long term particularly once we start on some of the larger changes we've
> been musing about for Jena 3 (e.g. module reorgs). Ideally the more
> disruptive module reorgs should go into Jena 3 ASAP and be included in the
> first 3.0.0 release
>
> Rob
Andy
>
> On 24/01/2015 10:33, "Andy Seaborne" <an...@apache.org> wrote:
>
>> Here is a suggestion for Jena3.
>>
>> After 2.12.2 or 2.12.3,
>>
>> 1/ A branch for Jena2 ; master is Jena3.
>> 2/ Switch to RDF 1.1 setting for strings
>> 3/ Rename com.hp.hpl.jena packages to org.apache.jena
>> 4/ Other changes
>> 5/ Let things settle down.
>> 6/ Release (beta? just go for it as 3.0.0?)
>>
>> (3) is the disruptive step - I doubt git merge is going to be much help
>> after that for managing changes related to com.hp.hpl.jena packages
>>
>> I wrote some migration notes for the RDF 1.1 isms and packages.
>>
>> http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>>
>> There is a lot of things that could be done. I like us to avoid
>> over-committing ourselves. The question I have is what is *necessary*
>> to drive into jena 3.first.
>>
>> Andy
>
>
>
>
Re: Jena3 planning
Posted by Rob Vesse <rv...@dotnetrdf.org>.
git merge is generally not too terrible at following file renames provided
you do them in a single git commit so it recognises that something was
renamed. However the package name changes (and thus the import statement
changes) may mean that cherry picking or merging across branches requires
a certain amount of manual fix up.
I guess the better question is "What is our support policy for Jena 2?"
I assume that there would need to be a transitional period during which
bug fixes go to both Jena 2 and 3 but in the longer term the intent would
be to get everyone to migrate to Jena 3 and then stop supporting Jena 2.
Since it would still exist as a branch people could choose to maintain it
themselves once the project dropped support if they really couldn't
migrate to Jena 2.
If Jena 3 is primarily the package rename and RDF 1.1 on by default then I
don't see any reason to use a Beta modifier. The essential code remains
the same and is well tested and deployed so the Beta moniker seems
inappropriate.
I read your migration instructions and think they are pretty clear and
concise. I hope that people won't have too many issues migrating forwards
to Jena 3 (famous last words!). I suspect the biggest problem will be
exactly the same as we see now, old code and tools out there referencing
old versions of the libraries and creating class path conflicts when users
(inadvertently or otherwise) end up mixing different library versions.
I think ultimately our long term strategy has to be "Please move to Jena
3" because we won't have the bandwidth to maintain two parallel branches
long term particularly once we start on some of the larger changes we've
been musing about for Jena 3 (e.g. module reorgs). Ideally the more
disruptive module reorgs should go into Jena 3 ASAP and be included in the
first 3.0.0 release
Rob
On 24/01/2015 10:33, "Andy Seaborne" <an...@apache.org> wrote:
>Here is a suggestion for Jena3.
>
>After 2.12.2 or 2.12.3,
>
>1/ A branch for Jena2 ; master is Jena3.
>2/ Switch to RDF 1.1 setting for strings
>3/ Rename com.hp.hpl.jena packages to org.apache.jena
>4/ Other changes
>5/ Let things settle down.
>6/ Release (beta? just go for it as 3.0.0?)
>
>(3) is the disruptive step - I doubt git merge is going to be much help
>after that for managing changes related to com.hp.hpl.jena packages
>
>I wrote some migration notes for the RDF 1.1 isms and packages.
>
>http://jena.staging.apache.org/documentation/migrate_jena2_jena3.html
>
>There is a lot of things that could be done. I like us to avoid
>over-committing ourselves. The question I have is what is *necessary*
>to drive into jena 3.first.
>
> Andy