You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Frank Scholten (Created) (JIRA)" <ji...@apache.org> on 2011/12/06 17:19:40 UTC
[jira] [Created] (MAHOUT-917) Build takes too long
Build takes too long
--------------------
Key: MAHOUT-917
URL: https://issues.apache.org/jira/browse/MAHOUT-917
Project: Mahout
Issue Type: Improvement
Components: build
Reporter: Frank Scholten
On my machine a full mvn clean install takes 55 minutes.
As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (MAHOUT-917) Build takes too long
Posted by Ted Dunning <te...@gmail.com>.
This is much less true when applied to map-reduce programs. The point of
scalability is that the code, well, scales. As such, with a few caveats
testing small is actually pretty similar to testing large. This isn't
quite try when it comes to convergence rates and such, but basic function
should still be there and can be confirmed without really huge scale
testing.
On Mon, Dec 12, 2011 at 8:21 PM, Lance Norskog <go...@gmail.com> wrote:
> If it is supposed to run huge, then it has to be tested huge. The
> current conversation not address this.
>
Re: [jira] [Commented] (MAHOUT-917) Build takes too long
Posted by Lance Norskog <go...@gmail.com>.
The current unit tests and small-scale end-to-end tests are fine.
Maybe a few could be trimmed down. Some of the long-running ones use
fair-sized datasets to verify that they crunch numbers correctly.
Mahout has some algorithms for which a realistic test should take a
few hours and several servers. Most of the effort in this project is
aimed at these implementations. I assume that the Apache Foundation
did not quite expect this level of machine support :) It would be
donated by companies with an interest in Mahout. Perhaps each company
could adopt one or two large-scale tests and donate server farms?
If it is supposed to run huge, then it has to be tested huge. The
current conversation not address this.
On Mon, Dec 12, 2011 at 12:37 PM, Isabel Drost <is...@apache.org> wrote:
> On 08.12.2011 Lance Norskog wrote:
>> Given all this (especially the third), a separate Maven sub-project makes
>> more sense; it should not be part of the default build.
>
> It may just be my personal preference, but I prefer having unit test accompany
> the maven module they test. That also makes checking coverage easier.
> Introducing a separate profile that is only executed on e.g. a nightly Jenkins
> build as suggested by Frank earlier is useful for that.
>
> Isabel
--
Lance Norskog
goksron@gmail.com
Re: [jira] [Commented] (MAHOUT-917) Build takes too long
Posted by Isabel Drost <is...@apache.org>.
On 08.12.2011 Lance Norskog wrote:
> Given all this (especially the third), a separate Maven sub-project makes
> more sense; it should not be part of the default build.
It may just be my personal preference, but I prefer having unit test accompany
the maven module they test. That also makes checking coverage easier.
Introducing a separate profile that is only executed on e.g. a nightly Jenkins
build as suggested by Frank earlier is useful for that.
Isabel
Re: [jira] [Commented] (MAHOUT-917) Build takes too long
Posted by Lance Norskog <go...@gmail.com>.
1) What does 'nightly' mean to 'mvn install'?
2) These algorithms are intended to run with full-size data, and their
output is approximate. There is no point in running them on toy data. They
should be run on real data with expected ranges of output, and sometimes
non-trivial code to check those outputs.
3) I may not want to download the real-size data they need.
Given all this (especially the third), a separate Maven sub-project makes
more sense; it should not be part of the default build.
On Thu, Dec 8, 2011 at 9:49 AM, Jake Mannix (Commented) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165354#comment-13165354]
>
> Jake Mannix commented on MAHOUT-917:
> ------------------------------------
>
> Yeah, the new LDA integration test is just slow. It runs on a *tiny* data
> set, but the problem is: to verify correctness, it needs to run an
> iterative map-reduce job first for numExpectedTopics - 1, then
> numExpectedTopics, then numExpectedTopics + 1, to verify that the
> perplexity is lowest for numExpectedTopics. Each one is hoping for
> convergence after only 5 iterations currently, but we're still talking
> about 15 map-reduce jobs launched from junit, for that one test. Not much
> that can be done to speed it up, but it could certainly be run as a
> @nightly.
>
> > Build takes too long
> > --------------------
> >
> > Key: MAHOUT-917
> > URL: https://issues.apache.org/jira/browse/MAHOUT-917
> > Project: Mahout
> > Issue Type: Improvement
> > Components: build
> > Reporter: Frank Scholten
> >
> > On my machine a full mvn clean install takes 55 minutes.
> > As an experiment I put all MapReduce job tests for all clustering
> algorithms on ignore. This reduces the build to 45 minutes. There are a lot
> of these long running tests in the project.
> > What about creating a separate maven profile for the nightly build that
> run all MapReduce job tests? For this we have to move these MapReduce tests
> > to separate classes with a naming convention such as *JobTest or
> *IntegrationTest and add some maven configuration.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
--
Lance Norskog
goksron@gmail.com
Re: [jira] [Commented] (MAHOUT-917) Build takes too long
Posted by Lance Norskog <go...@gmail.com>.
mvn -DskipTests=true clean install
It's only a few minutes on my 3-yr-old laptop. The tests should be split
into 'unit' (smoke) tests and numerical quality tests. The latter are what
take the time, and some have been 'compromised' in order to fit the unit
test mold. I think they should all run as a separate batch, or even
module, and is allowed, nay encouraged, to use real data and take many
hours. Some projects have regressed in quality (Bayes for example) without
anyone noticing.
On Tue, Dec 6, 2011 at 1:27 PM, Sean Owen (Commented) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163866#comment-13163866]
>
> Sean Owen commented on MAHOUT-917:
> ----------------------------------
>
> I don't know if it's so hard... a few tests have been checked in that take
> over 30 minutes to run (one in the last week!) that I 'fixed' relatively
> easily. They were running on a test input that was perhaps 100x bigger than
> it needed to be.
>
> If you run, you'll see a handful of tests taking well over half that time.
> I think it'd be awful nice if the authors went back and tried to just run a
> smaller test. Or disable them.
>
> In fact I'd be fine with: if it takes more than 3 minutes to run it should
> be disabled until the contributor can fix it.
>
> > Build takes too long
> > --------------------
> >
> > Key: MAHOUT-917
> > URL: https://issues.apache.org/jira/browse/MAHOUT-917
> > Project: Mahout
> > Issue Type: Improvement
> > Components: build
> > Reporter: Frank Scholten
> >
> > On my machine a full mvn clean install takes 55 minutes.
> > As an experiment I put all MapReduce job tests for all clustering
> algorithms on ignore. This reduces the build to 45 minutes. There are a lot
> of these long running tests in the project.
> > What about creating a separate maven profile for the nightly build that
> run all MapReduce job tests? For this we have to move these MapReduce tests
> > to separate classes with a naming convention such as *JobTest or
> *IntegrationTest and add some maven configuration.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
--
Lance Norskog
goksron@gmail.com
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399263#comment-13399263 ]
Sebastian Schelter commented on MAHOUT-917:
-------------------------------------------
I'm not sure this is really an issue. You can always run particular tests from the IDE or on the commandline via mvn. To simply build or check for compilation errors, one can build using -DskipTests.
Waiting until the tests finish before you commit or upload a patch is not a big hurdle in my eyes. I see it as a good thing that we have a huge number of intensive tests.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
> Assignee: Sebastian Schelter
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165207#comment-13165207 ]
Grant Ingersoll commented on MAHOUT-917:
----------------------------------------
https://builds.apache.org/job/Mahout-Quality/1237/testReport/junit/
Vectorizer and SSVD seem to be fairly big culprits, as well as kmeans, new LDA and some of the Hadoop item stuff.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Isabel Drost (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163857#comment-13163857 ]
Isabel Drost commented on MAHOUT-917:
-------------------------------------
As much as I would like to see those long running tests optimised I think that is not going to happen any time soon. So I am very much in favour of your proposed solution.
One warning: I already hear complaints from those using an IDE to run the whole test suite by right clicking the test folder that all tests are being executed. Being one of those seemingly rare human beings who mostly interacts with maven from the command line I do not care personally.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Sean Owen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163866#comment-13163866 ]
Sean Owen commented on MAHOUT-917:
----------------------------------
I don't know if it's so hard... a few tests have been checked in that take over 30 minutes to run (one in the last week!) that I 'fixed' relatively easily. They were running on a test input that was perhaps 100x bigger than it needed to be.
If you run, you'll see a handful of tests taking well over half that time. I think it'd be awful nice if the authors went back and tried to just run a smaller test. Or disable them.
In fact I'd be fine with: if it takes more than 3 minutes to run it should be disabled until the contributor can fix it.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAHOUT-917) Build takes too long
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter reassigned MAHOUT-917:
-----------------------------------------
Assignee: Sebastian Schelter
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
> Assignee: Sebastian Schelter
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164380#comment-13164380 ]
Grant Ingersoll commented on MAHOUT-917:
----------------------------------------
I think we should make MahoutTestCase extend Lucene's test-framework. In there, we have annotations such as @slow, @nightly, @weekly, etc. which can then be controlled at run time via properties mvn test -Dtests.nightly=true, for instance. We use this to segment out various tests (we have one test that takes ~4 hours to run)
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165217#comment-13165217 ]
Grant Ingersoll commented on MAHOUT-917:
----------------------------------------
Seems a lot of the overhead is simply due to Hadoop, too.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Ted Dunning (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165877#comment-13165877 ]
Ted Dunning commented on MAHOUT-917:
------------------------------------
I think that the vectorizer tests could be augmented with simpler tests that just exercise the mapper and leave the actual running of a map-reduce program to an "expensive test" category. That should drop these tests into the millisecond range.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Jake Mannix (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165354#comment-13165354 ]
Jake Mannix commented on MAHOUT-917:
------------------------------------
Yeah, the new LDA integration test is just slow. It runs on a *tiny* data set, but the problem is: to verify correctness, it needs to run an iterative map-reduce job first for numExpectedTopics - 1, then numExpectedTopics, then numExpectedTopics + 1, to verify that the perplexity is lowest for numExpectedTopics. Each one is hoping for convergence after only 5 iterations currently, but we're still talking about 15 map-reduce jobs launched from junit, for that one test. Not much that can be done to speed it up, but it could certainly be run as a @nightly.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399264#comment-13399264 ]
Sebastian Schelter commented on MAHOUT-917:
-------------------------------------------
I'd vote for 'Not a Problem' on this, I think there are more pressing issues currently.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
> Assignee: Sebastian Schelter
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165208#comment-13165208 ]
Grant Ingersoll commented on MAHOUT-917:
----------------------------------------
We really should solve MAHOUT-916 and see how that looks.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-917) Build takes too long
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164382#comment-13164382 ]
Grant Ingersoll commented on MAHOUT-917:
----------------------------------------
But, also agree w/ Sean, we should look to cut the times.
> Build takes too long
> --------------------
>
> Key: MAHOUT-917
> URL: https://issues.apache.org/jira/browse/MAHOUT-917
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Reporter: Frank Scholten
>
> On my machine a full mvn clean install takes 55 minutes.
> As an experiment I put all MapReduce job tests for all clustering algorithms on ignore. This reduces the build to 45 minutes. There are a lot of these long running tests in the project.
> What about creating a separate maven profile for the nightly build that run all MapReduce job tests? For this we have to move these MapReduce tests
> to separate classes with a naming convention such as *JobTest or *IntegrationTest and add some maven configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira