You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by "Jukka Zitting (JIRA)" <ji...@apache.org> on 2012/07/03 15:01:19 UTC
[jira] [Created] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Jukka Zitting created OAK-163:
---------------------------------
Summary: Move the JCR TCK back to the integrationTesting profile
Key: OAK-163
URL: https://issues.apache.org/jira/browse/OAK-163
Project: Jackrabbit Oak
Issue Type: Improvement
Components: jcr
Reporter: Jukka Zitting
Priority: Minor
In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406401#comment-13406401 ]
Jukka Zitting commented on OAK-163:
-----------------------------------
bq. There's a risk that people won't run the TCK before checking in, causing loss of cycles for somebody else later on.
That's what we have the CI build for.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406432#comment-13406432 ]
Michael Dürig commented on OAK-163:
-----------------------------------
bq. There's a risk that people won't run the TCK before checking in, causing loss of cycles for somebody else later on.
In my experience this is not only a risk but common practice. And the CI server doesn't seem to help either. http://ci.apache.org/builders/oak-trunk fails since quite a while now and no one bothers to have a look.
Having long build times is inconvenient agreed. But failing the build for all others is a very bad trade off. Optimally:
* everyone runs the integration tests before checkin unless there is very good reasons to believe nothing will break
* the CI infrastructure would be reliable
* everyone watches the CI server and fixes broken builds caused by self.
Until we have that, I suggest we'd turn the process around and add an option to smoke test like we had it in OAK-53 before it was reverted.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting resolved OAK-163.
-------------------------------
Resolution: Fixed
Fix Version/s: 0.4
Assignee: Jukka Zitting
Noting the above objections, I still decided to implement this change in revision 1359014. It's possible to switch back to the previous build behavior by setting the OAK_INTEGRATION_TESTING environment variable in your environment, and we can always move the TCK back to the default build if we start having trouble with broken builds because of this.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Assignee: Jukka Zitting
> Priority: Minor
> Labels: tck
> Fix For: 0.4
>
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Julian Reschke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406398#comment-13406398 ]
Julian Reschke commented on OAK-163:
------------------------------------
There's a risk that people *won't* run the TCK before checking in, causing loss of cycles for somebody else later on.
Also, running with integrationTesting is so slow for jackrabbit that most people won't wait for it to finish before checking in. So what may be right for OAK definitively is not right for Jackrabbit.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated OAK-163:
------------------------------
Attachment: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
The attached patch implements the proposed change.
Relevant build times (on my computer) after the patch is applied:
* 1:22 for {{mvn clean install -DskipTests}}
* 2:01 for {{mvn clean install}}
* 6:01 for {{mvn clean install -PintegrationTesting}}
Build time without the patch:
* 5:01 for {{mvn clean install}}
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Julian Reschke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406419#comment-13406419 ]
Julian Reschke commented on OAK-163:
------------------------------------
bq. That's what we have the CI build for.
Downsides: Hasn't been always reliable. Problem is found only after checkin. Developer might not see it in time (end-of-day, vacation) or not paying attention. Etc.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406438#comment-13406438 ]
Jukka Zitting commented on OAK-163:
-----------------------------------
bq. http://ci.apache.org/builders/oak-trunk fails since quite a while now and no one bothers to have a look.
Not true. I caught that one right after the CI build stated failing: http://markmail.org/message/zhg7ilg7a2dg7s4z.
Of course the issue with Buildbot still isn't resolved, but we have the Travis build as a backup.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406377#comment-13406377 ]
Jukka Zitting commented on OAK-163:
-----------------------------------
I'd like to change that in Jackrabbit trunk as well, there the default build already takes >10 minutes for me. That's enough time for a coffee break for each build of the codebase.
A long build time is a drain on productivity as to get a deployable build artifact (runnable jar, OSGi bundle, etc.) you need to run the build. When working on some non-trivial issue, I typically have to build the codebase at least a half a dozen times before the changes are ready to be committed. There's little point having long-running tests included in such intermediate builds, but it would still be useful to have a basic set of fast and simple sanity checks run on the code to catch any obvious mistakes beyond just compile failures.
Currently I'm using {{mvn clean install -DskipTests}} for such intermediate builds, which is hardly ideal as a) that disables all tests, and b) requires more typing than the default build.
Instead I'd like the default build (that's executed most often) to be lean and fast (with unit tests ideally not taking much more time than it took to compile the relevant code), and have a separate integration test build ({{mvn clean install -PintegrationTesting}}) for more complete verification of changes before committing them.
If people prefer to have all tests run on each build, we could still support that use case on a per-user basis by having the {{integrationTesting}} profile be automatically activated on something like the presence of an {{OAK_INTEGRATION_TESTING}} environment variable.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Julian Reschke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405916#comment-13405916 ]
Julian Reschke commented on OAK-163:
------------------------------------
Not convinced; we run them as part of the default jackrabbit build as well.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-163) Move the JCR TCK back to the
integrationTesting profile
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406436#comment-13406436 ]
Jukka Zitting commented on OAK-163:
-----------------------------------
bq. Hasn't been always reliable.
People are still expected to run the TCK before committing new stuff, so the CI build is just a failsafe for cases when that didn't happen. And if the CI build also fails to catch the problem, other committers will soon enough encounter the build failure. A test failure won't last long before being detected.
bq. Developer might not see it in time (end-of-day, vacation) or not paying attention.
We manage the codebase collectively, so anyone who sees a build failure is free to revert the commit that broke the build. Personally I much rather spend a few extra minutes reverting the occasional breaking change than having all my builds take longer.
I'd also be fine with a solution that I can selectively enable only on my machine to keep only fast tests in the default build, but I don't know a clean way of doing that. It's easier to selectively add bits to the build than to take them away.
See also OAK-53 where we already discussed the "long build" problem earlier.
> Move the JCR TCK back to the integrationTesting profile
> -------------------------------------------------------
>
> Key: OAK-163
> URL: https://issues.apache.org/jira/browse/OAK-163
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
> Reporter: Jukka Zitting
> Priority: Minor
> Labels: tck
> Attachments: 0001-OAK-163-Move-the-JCR-TCK-back-to-the-integrationTest.patch
>
>
> In revision 1325811 the JCR TCK tests were moved from the integrationTesting profile to a normal build. However, since then the execution time of the TCK has grown to 3+ minutes, which is more than the rest of the Oak build combined. To keep the build time down to at most a couple of minutes, I'd like to move the TCK run back to the integrationTesting profile where it will get executed only when explicitly requested (typically manually before a commit or after one in a CI build).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira