You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by pi song <pi...@gmail.com> on 2008/05/12 15:56:35 UTC
Slow unit testing
It has been a while since we moved from Hadoop15 to Hadoop16 that all the
MapReduce unit tests run significantly slower than before (at least 2-3x).
What is the reason behind this? Is there a way to fix?
Since everyone has to run at least a full test before submitting a patch.
This is not productive from developer's point of view.
Pi
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
That does not sound right. Did yiu check the logs to see if there are
any errors?
> -----Original Message-----
> From: pi song [mailto:pi.songs@gmail.com]
> Sent: Monday, May 12, 2008 7:15 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Slow unit testing
>
> Already more than 40 minutes.
>
> On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich <ol...@yahoo-inc.com>
> wrote:
>
> > I run the tests on the regular basis and don't see a
> proble. MR tests
> > take about 20 minites for me. How long does it take for you?
> >
> > Olga
> >
> > > -----Original Message-----
> > > From: pi song [mailto:pi.songs@gmail.com]
> > > Sent: Monday, May 12, 2008 6:57 AM
> > > To: pig-dev@incubator.apache.org
> > > Subject: Slow unit testing
> > >
> > > It has been a while since we moved from Hadoop15 to Hadoop16 that
> > > all the MapReduce unit tests run significantly slower than before
> > > (at least 2-3x).
> > > What is the reason behind this? Is there a way to fix?
> > >
> > > Since everyone has to run at least a full test before
> submitting a
> > > patch.
> > > This is not productive from developer's point of view.
> > >
> > > Pi
> > >
> >
>
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
I don't think unit tests make connection to hod. Also, I don't see the
slowdown when I run the tests with hadoop 0.16
> -----Original Message-----
> From: pi song [mailto:pi.songs@gmail.com]
> Sent: Monday, May 12, 2008 7:20 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Slow unit testing
>
> This started to happen since the introduction of Hadoop16
> (which includes changes in HOD). The time shown in ANT-Junit
> is not accurate. What I can see here is that time spent
> doing nothing between each tests is long but not taken into account.
>
> Is it possible to be something to do with connecting to HOD?
>
> [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> elapsed: 262.014 sec
> [junit] Running org.apache.pig.test.TestFilterOpString
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> elapsed: 234.839 sec
> [junit] Running org.apache.pig.test.TestGrunt
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.159 sec
>
> Pi
>
> On Tue, May 13, 2008 at 12:15 AM, pi song <pi...@gmail.com> wrote:
>
> > Already more than 40 minutes.
> >
> >
> > On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich
> <ol...@yahoo-inc.com>
> > wrote:
> >
> > > I run the tests on the regular basis and don't see a proble. MR
> > > tests take about 20 minites for me. How long does it take for you?
> > >
> > > Olga
> > >
> > > > -----Original Message-----
> > > > From: pi song [mailto:pi.songs@gmail.com]
> > > > Sent: Monday, May 12, 2008 6:57 AM
> > > > To: pig-dev@incubator.apache.org
> > > > Subject: Slow unit testing
> > > >
> > > > It has been a while since we moved from Hadoop15 to
> Hadoop16 that
> > > > all the MapReduce unit tests run significantly slower
> than before
> > > > (at least 2-3x).
> > > > What is the reason behind this? Is there a way to fix?
> > > >
> > > > Since everyone has to run at least a full test before
> submitting a
> > > > patch.
> > > > This is not productive from developer's point of view.
> > > >
> > > > Pi
> > > >
> > >
> >
> >
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
FYI,
Starting Jetty in Hadoop on my machine takes much longer than you.
Normally, it takes at least 15 seconds each time here when it takes
sub-second in your logs.
I will have to dig more.
08/05/10 23:22:01 INFO fs.FSNamesystem: Registered FSNamesystemStatusMBean
08/05/10 23:22:16 INFO util.Credential: Checking Resource aliases
08/05/10 23:22:16 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:22:17 INFO dfs.DataNode: Balancing bandwith is 1048576 bytes/s
08/05/10 23:22:32 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:22:33 INFO util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@147358f
08/05/10 23:22:33 INFO dfs.DataNode: BlockReport of 0 blocks got processed
in 5 msecs
08/05/10 23:22:48 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:22:48 INFO util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@1d87b85
08/05/10 23:22:49 INFO dfs.DataNode: BlockReport of 0 blocks got processed
in 6 msecs
08/05/10 23:23:04 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:23:04 INFO util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@1412e75
08/05/10 23:23:04 INFO dfs.DataNode: BlockReport of 0 blocks got processed
in 8 msecs
08/05/10 23:23:19 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:23:20 INFO util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@15d252d
08/05/10 23:23:36 INFO mapred.MiniMRCluster: mapred.local.dir is
/tmp/hadoop-root/mapred/local/0_0
08/05/10 23:23:51 INFO http.HttpServer: Version Jetty/5.1.x
08/05/10 23:23:51 INFO util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@903025
On Tue, May 13, 2008 at 8:54 AM, pi song <pi...@gmail.com> wrote:
> Thanks for your information.
> I'll look at this tonight.
>
>
> On 5/13/08, Olga Natkovich <ol...@yahoo-inc.com> wrote:
> >
> > We do see similar slowdown testing with hadoop 0.17 and still trying to
> > figure out why this is happening. I wonder if this could be related.
> >
> > > -----Original Message-----
> > > From: Alan Gates [mailto:gates@yahoo-inc.com]
> > > Sent: Monday, May 12, 2008 9:41 AM
> > > To: pig-dev@incubator.apache.org
> > > Subject: Re: Slow unit testing
> > >
> > > Sorry, I should have been more clear. Time results similar
> > > to Olga's, not Pi's.
> > >
> > > Alan.
> > >
> > > Alan Gates wrote:
> > > > I run the unit tests on a mac laptop and am seeing similar
> > > time results.
> > > >
> > > > Alan.
> > > >
> > > > Olga Natkovich wrote:
> > > >> Pi,
> > > >>
> > > >> Inline my stdout content. Attached test output files. I ran this
> > > >> against latest trunk on a linux box.
> > > >>
> > > >> test-local:
> > > >>
> > > >> testOneMode:
> > > >> [mkdir] Created dir:
> > > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
> > > >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time
> > > elapsed: 2.93 sec
> > > >> [junit] Running org.apache.pig.test.TestBinaryStorage
> > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > > elapsed: 0.662
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestBuiltin
> > > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> > > >> 3.849 sec
> > > >> [junit] Running org.apache.pig.test.TestCmdLineParser
> > > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> > > >> 0.011 sec
> > > >> [junit] Running org.apache.pig.test.TestCombiner
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.421
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestCompressedFiles
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.337
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestCustomSlicer
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.214
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestDataBag
> > > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> > > >> 1.461 sec
> > > >> [junit] Running org.apache.pig.test.TestDataModel
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > > elapsed: 0.126
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestEvalPipeline
> > > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time
> > > elapsed: 5.035
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestExGenCogroup
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > > elapsed: 1.246
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestExGenEval
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 1.172
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> > > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> > > elapsed: 1.76 sec
> > > >> [junit] Running org.apache.pig.test.TestFilterOpString
> > > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> > > elapsed: 1.385
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestGrunt
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.238
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > > >> 57.608 sec
> > > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> > > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time
> > > elapsed: 1.04
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestMapReduce
> > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > > >> 73.303 sec
> > > >> [junit] Running
> > > org.apache.pig.test.TestMapReduceResultRecycling
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 1.542
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> > > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> > > >> 0.633 sec
> > > >> [junit] Running org.apache.pig.test.TestParser
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.206
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestPigContext
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 0.806
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestPigFile
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 3.344
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestPigScriptParser
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.441
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestPigServer
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > > elapsed: 5.76 sec
> > > >> [junit] Running org.apache.pig.test.TestPigSplit
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.546
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > > elapsed: 0.622
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestStore
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 2.139
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestStreaming
> > > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> > > >> 1.074 sec
> > > >>
> > > >> test-mapreduce:
> > > >>
> > > >> testOneMode:
> > > >> [mkdir] Created dir:
> > > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
> > > >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > > >> 74.738 sec
> > > >> [junit] Running org.apache.pig.test.TestBinaryStorage
> > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > > elapsed: 36.99
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestBuiltin
> > > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> > > >> 4.513 sec
> > > >> [junit] Running org.apache.pig.test.TestCmdLineParser
> > > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> > > >> 0.034 sec
> > > >> [junit] Running org.apache.pig.test.TestCombiner
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 24.9 sec
> > > >> [junit] Running org.apache.pig.test.TestCompressedFiles
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > > >> 29.658 sec
> > > >> [junit] Running org.apache.pig.test.TestCustomSlicer
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > > >> 90.949 sec
> > > >> [junit] Running org.apache.pig.test.TestDataBag
> > > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> > > >> 1.605 sec
> > > >> [junit] Running org.apache.pig.test.TestDataModel
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > > elapsed: 0.194
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestEvalPipeline
> > > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed:
> > > >> 203.826 sec
> > > >> [junit] Running org.apache.pig.test.TestExGenCogroup
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > > >> 15.116 sec
> > > >> [junit] Running org.apache.pig.test.TestExGenEval
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 8.609
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> > > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed:
> > > >> 104.898 sec
> > > >> [junit] Running org.apache.pig.test.TestFilterOpString
> > > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:
> > > >> 83.079 sec
> > > >> [junit] Running org.apache.pig.test.TestGrunt
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.296
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > > >> 50.802 sec
> > > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> > > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed:
> > > >> 1.328 sec
> > > >> [junit] Running org.apache.pig.test.TestMapReduce
> > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > > >> 52.169 sec
> > > >> [junit] Running
> > > org.apache.pig.test.TestMapReduceResultRecycling
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > > >> 43.827 sec
> > > >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> > > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> > > >> 0.724 sec
> > > >> [junit] Running org.apache.pig.test.TestParser
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 8.08 sec
> > > >> [junit] Running org.apache.pig.test.TestPigContext
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 1.209
> > > >> sec
> > > >> [junit] Running org.apache.pig.test.TestPigFile
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > > elapsed: 0.95 sec
> > > >> [junit] Running org.apache.pig.test.TestPigScriptParser
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > > elapsed: 0.26 sec
> > > >> [junit] Running org.apache.pig.test.TestPigServer
> > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > > >> 10.313 sec
> > > >> [junit] Running org.apache.pig.test.TestPigSplit
> > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > > >> 109.971 sec
> > > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
> > > >> 11.803 sec
> > > >> [junit] Running org.apache.pig.test.TestStore
> > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed:
> > > >> 125.27 sec
> > > >> [junit] Running org.apache.pig.test.TestStreaming
> > > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> > > >> 142.835 sec
> > > >>
> > > >> test:
> > > >>
> > > >> BUILD SUCCESSFUL
> > > >> Total time: 26 minutes 4 seconds
> > > >> ucdev3:~/src/pig-apache3/trunk> cd
> > > >> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
> > >
> >
>
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
Thanks for your information.
I'll look at this tonight.
On 5/13/08, Olga Natkovich <ol...@yahoo-inc.com> wrote:
>
> We do see similar slowdown testing with hadoop 0.17 and still trying to
> figure out why this is happening. I wonder if this could be related.
>
> > -----Original Message-----
> > From: Alan Gates [mailto:gates@yahoo-inc.com]
> > Sent: Monday, May 12, 2008 9:41 AM
> > To: pig-dev@incubator.apache.org
> > Subject: Re: Slow unit testing
> >
> > Sorry, I should have been more clear. Time results similar
> > to Olga's, not Pi's.
> >
> > Alan.
> >
> > Alan Gates wrote:
> > > I run the unit tests on a mac laptop and am seeing similar
> > time results.
> > >
> > > Alan.
> > >
> > > Olga Natkovich wrote:
> > >> Pi,
> > >>
> > >> Inline my stdout content. Attached test output files. I ran this
> > >> against latest trunk on a linux box.
> > >>
> > >> test-local:
> > >>
> > >> testOneMode:
> > >> [mkdir] Created dir:
> > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
> > >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time
> > elapsed: 2.93 sec
> > >> [junit] Running org.apache.pig.test.TestBinaryStorage
> > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > elapsed: 0.662
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestBuiltin
> > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> > >> 3.849 sec
> > >> [junit] Running org.apache.pig.test.TestCmdLineParser
> > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> > >> 0.011 sec
> > >> [junit] Running org.apache.pig.test.TestCombiner
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.421
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestCompressedFiles
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.337
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestCustomSlicer
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.214
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestDataBag
> > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> > >> 1.461 sec
> > >> [junit] Running org.apache.pig.test.TestDataModel
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > elapsed: 0.126
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestEvalPipeline
> > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time
> > elapsed: 5.035
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestExGenCogroup
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > elapsed: 1.246
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestExGenEval
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 1.172
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> > elapsed: 1.76 sec
> > >> [junit] Running org.apache.pig.test.TestFilterOpString
> > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> > elapsed: 1.385
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestGrunt
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.238
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > >> 57.608 sec
> > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time
> > elapsed: 1.04
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestMapReduce
> > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > >> 73.303 sec
> > >> [junit] Running
> > org.apache.pig.test.TestMapReduceResultRecycling
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 1.542
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> > >> 0.633 sec
> > >> [junit] Running org.apache.pig.test.TestParser
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.206
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestPigContext
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 0.806
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestPigFile
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 3.344
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestPigScriptParser
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.441
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestPigServer
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > elapsed: 5.76 sec
> > >> [junit] Running org.apache.pig.test.TestPigSplit
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.546
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > elapsed: 0.622
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestStore
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 2.139
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestStreaming
> > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> > >> 1.074 sec
> > >>
> > >> test-mapreduce:
> > >>
> > >> testOneMode:
> > >> [mkdir] Created dir:
> > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
> > >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > >> 74.738 sec
> > >> [junit] Running org.apache.pig.test.TestBinaryStorage
> > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> > elapsed: 36.99
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestBuiltin
> > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> > >> 4.513 sec
> > >> [junit] Running org.apache.pig.test.TestCmdLineParser
> > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> > >> 0.034 sec
> > >> [junit] Running org.apache.pig.test.TestCombiner
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 24.9 sec
> > >> [junit] Running org.apache.pig.test.TestCompressedFiles
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > >> 29.658 sec
> > >> [junit] Running org.apache.pig.test.TestCustomSlicer
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > >> 90.949 sec
> > >> [junit] Running org.apache.pig.test.TestDataBag
> > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> > >> 1.605 sec
> > >> [junit] Running org.apache.pig.test.TestDataModel
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> > elapsed: 0.194
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestEvalPipeline
> > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed:
> > >> 203.826 sec
> > >> [junit] Running org.apache.pig.test.TestExGenCogroup
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > >> 15.116 sec
> > >> [junit] Running org.apache.pig.test.TestExGenEval
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 8.609
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed:
> > >> 104.898 sec
> > >> [junit] Running org.apache.pig.test.TestFilterOpString
> > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:
> > >> 83.079 sec
> > >> [junit] Running org.apache.pig.test.TestGrunt
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.296
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > >> 50.802 sec
> > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed:
> > >> 1.328 sec
> > >> [junit] Running org.apache.pig.test.TestMapReduce
> > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> > >> 52.169 sec
> > >> [junit] Running
> > org.apache.pig.test.TestMapReduceResultRecycling
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > >> 43.827 sec
> > >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> > >> 0.724 sec
> > >> [junit] Running org.apache.pig.test.TestParser
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 8.08 sec
> > >> [junit] Running org.apache.pig.test.TestPigContext
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 1.209
> > >> sec
> > >> [junit] Running org.apache.pig.test.TestPigFile
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> > elapsed: 0.95 sec
> > >> [junit] Running org.apache.pig.test.TestPigScriptParser
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.26 sec
> > >> [junit] Running org.apache.pig.test.TestPigServer
> > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> > >> 10.313 sec
> > >> [junit] Running org.apache.pig.test.TestPigSplit
> > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> > >> 109.971 sec
> > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
> > >> 11.803 sec
> > >> [junit] Running org.apache.pig.test.TestStore
> > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed:
> > >> 125.27 sec
> > >> [junit] Running org.apache.pig.test.TestStreaming
> > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> > >> 142.835 sec
> > >>
> > >> test:
> > >>
> > >> BUILD SUCCESSFUL
> > >> Total time: 26 minutes 4 seconds
> > >> ucdev3:~/src/pig-apache3/trunk> cd
> > >> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
> >
>
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
We do see similar slowdown testing with hadoop 0.17 and still trying to
figure out why this is happening. I wonder if this could be related.
> -----Original Message-----
> From: Alan Gates [mailto:gates@yahoo-inc.com]
> Sent: Monday, May 12, 2008 9:41 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Slow unit testing
>
> Sorry, I should have been more clear. Time results similar
> to Olga's, not Pi's.
>
> Alan.
>
> Alan Gates wrote:
> > I run the unit tests on a mac laptop and am seeing similar
> time results.
> >
> > Alan.
> >
> > Olga Natkovich wrote:
> >> Pi,
> >>
> >> Inline my stdout content. Attached test output files. I ran this
> >> against latest trunk on a linux box.
> >>
> >> test-local:
> >>
> >> testOneMode:
> >> [mkdir] Created dir:
> >> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
> >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time
> elapsed: 2.93 sec
> >> [junit] Running org.apache.pig.test.TestBinaryStorage
> >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> elapsed: 0.662
> >> sec
> >> [junit] Running org.apache.pig.test.TestBuiltin
> >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> >> 3.849 sec
> >> [junit] Running org.apache.pig.test.TestCmdLineParser
> >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> >> 0.011 sec
> >> [junit] Running org.apache.pig.test.TestCombiner
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.421
> >> sec
> >> [junit] Running org.apache.pig.test.TestCompressedFiles
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.337
> >> sec
> >> [junit] Running org.apache.pig.test.TestCustomSlicer
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.214
> >> sec
> >> [junit] Running org.apache.pig.test.TestDataBag
> >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> >> 1.461 sec
> >> [junit] Running org.apache.pig.test.TestDataModel
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> elapsed: 0.126
> >> sec
> >> [junit] Running org.apache.pig.test.TestEvalPipeline
> >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time
> elapsed: 5.035
> >> sec
> >> [junit] Running org.apache.pig.test.TestExGenCogroup
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> elapsed: 1.246
> >> sec
> >> [junit] Running org.apache.pig.test.TestExGenEval
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 1.172
> >> sec
> >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> elapsed: 1.76 sec
> >> [junit] Running org.apache.pig.test.TestFilterOpString
> >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> elapsed: 1.385
> >> sec
> >> [junit] Running org.apache.pig.test.TestGrunt
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.238
> >> sec
> >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> >> 57.608 sec
> >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time
> elapsed: 1.04
> >> sec
> >> [junit] Running org.apache.pig.test.TestMapReduce
> >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> >> 73.303 sec
> >> [junit] Running
> org.apache.pig.test.TestMapReduceResultRecycling
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 1.542
> >> sec
> >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> >> 0.633 sec
> >> [junit] Running org.apache.pig.test.TestParser
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.206
> >> sec
> >> [junit] Running org.apache.pig.test.TestPigContext
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 0.806
> >> sec
> >> [junit] Running org.apache.pig.test.TestPigFile
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 3.344
> >> sec
> >> [junit] Running org.apache.pig.test.TestPigScriptParser
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.441
> >> sec
> >> [junit] Running org.apache.pig.test.TestPigServer
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> elapsed: 5.76 sec
> >> [junit] Running org.apache.pig.test.TestPigSplit
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.546
> >> sec
> >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> elapsed: 0.622
> >> sec
> >> [junit] Running org.apache.pig.test.TestStore
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 2.139
> >> sec
> >> [junit] Running org.apache.pig.test.TestStreaming
> >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> >> 1.074 sec
> >>
> >> test-mapreduce:
> >>
> >> testOneMode:
> >> [mkdir] Created dir:
> >> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
> >> [junit] Running org.apache.pig.test.TestAlgebraicEval
> >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> >> 74.738 sec
> >> [junit] Running org.apache.pig.test.TestBinaryStorage
> >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time
> elapsed: 36.99
> >> sec
> >> [junit] Running org.apache.pig.test.TestBuiltin
> >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed:
> >> 4.513 sec
> >> [junit] Running org.apache.pig.test.TestCmdLineParser
> >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed:
> >> 0.034 sec
> >> [junit] Running org.apache.pig.test.TestCombiner
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 24.9 sec
> >> [junit] Running org.apache.pig.test.TestCompressedFiles
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> >> 29.658 sec
> >> [junit] Running org.apache.pig.test.TestCustomSlicer
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> >> 90.949 sec
> >> [junit] Running org.apache.pig.test.TestDataBag
> >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed:
> >> 1.605 sec
> >> [junit] Running org.apache.pig.test.TestDataModel
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time
> elapsed: 0.194
> >> sec
> >> [junit] Running org.apache.pig.test.TestEvalPipeline
> >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed:
> >> 203.826 sec
> >> [junit] Running org.apache.pig.test.TestExGenCogroup
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> >> 15.116 sec
> >> [junit] Running org.apache.pig.test.TestExGenEval
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 8.609
> >> sec
> >> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed:
> >> 104.898 sec
> >> [junit] Running org.apache.pig.test.TestFilterOpString
> >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:
> >> 83.079 sec
> >> [junit] Running org.apache.pig.test.TestGrunt
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.296
> >> sec
> >> [junit] Running org.apache.pig.test.TestInfixArithmetic
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> >> 50.802 sec
> >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed:
> >> 1.328 sec
> >> [junit] Running org.apache.pig.test.TestMapReduce
> >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
> >> 52.169 sec
> >> [junit] Running
> org.apache.pig.test.TestMapReduceResultRecycling
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> >> 43.827 sec
> >> [junit] Running org.apache.pig.test.TestParamSubPreproc
> >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed:
> >> 0.724 sec
> >> [junit] Running org.apache.pig.test.TestParser
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 8.08 sec
> >> [junit] Running org.apache.pig.test.TestPigContext
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 1.209
> >> sec
> >> [junit] Running org.apache.pig.test.TestPigFile
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time
> elapsed: 0.95 sec
> >> [junit] Running org.apache.pig.test.TestPigScriptParser
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.26 sec
> >> [junit] Running org.apache.pig.test.TestPigServer
> >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
> >> 10.313 sec
> >> [junit] Running org.apache.pig.test.TestPigSplit
> >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
> >> 109.971 sec
> >> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
> >> 11.803 sec
> >> [junit] Running org.apache.pig.test.TestStore
> >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed:
> >> 125.27 sec
> >> [junit] Running org.apache.pig.test.TestStreaming
> >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
> >> 142.835 sec
> >>
> >> test:
> >>
> >> BUILD SUCCESSFUL
> >> Total time: 26 minutes 4 seconds
> >> ucdev3:~/src/pig-apache3/trunk> cd
> >> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
>
Re: Slow unit testing
Posted by Alan Gates <ga...@yahoo-inc.com>.
Sorry, I should have been more clear. Time results similar to Olga's,
not Pi's.
Alan.
Alan Gates wrote:
> I run the unit tests on a mac laptop and am seeing similar time results.
>
> Alan.
>
> Olga Natkovich wrote:
>> Pi,
>>
>> Inline my stdout content. Attached test output files. I ran this against
>> latest trunk on a linux box.
>>
>> test-local:
>>
>> testOneMode:
>> [mkdir] Created dir:
>> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
>> [junit] Running org.apache.pig.test.TestAlgebraicEval
>> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.93 sec
>> [junit] Running org.apache.pig.test.TestBinaryStorage
>> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.662
>> sec
>> [junit] Running org.apache.pig.test.TestBuiltin
>> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 3.849
>> sec
>> [junit] Running org.apache.pig.test.TestCmdLineParser
>> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.011
>> sec
>> [junit] Running org.apache.pig.test.TestCombiner
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.421
>> sec
>> [junit] Running org.apache.pig.test.TestCompressedFiles
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.337
>> sec
>> [junit] Running org.apache.pig.test.TestCustomSlicer
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.214
>> sec
>> [junit] Running org.apache.pig.test.TestDataBag
>> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.461
>> sec
>> [junit] Running org.apache.pig.test.TestDataModel
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.126
>> sec
>> [junit] Running org.apache.pig.test.TestEvalPipeline
>> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 5.035
>> sec
>> [junit] Running org.apache.pig.test.TestExGenCogroup
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.246
>> sec
>> [junit] Running org.apache.pig.test.TestExGenEval
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.172
>> sec
>> [junit] Running org.apache.pig.test.TestFilterOpNumeric
>> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 1.76 sec
>> [junit] Running org.apache.pig.test.TestFilterOpString
>> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.385
>> sec
>> [junit] Running org.apache.pig.test.TestGrunt
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.238
>> sec
>> [junit] Running org.apache.pig.test.TestInfixArithmetic
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 57.608
>> sec
>> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
>> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.04
>> sec
>> [junit] Running org.apache.pig.test.TestMapReduce
>> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 73.303
>> sec
>> [junit] Running org.apache.pig.test.TestMapReduceResultRecycling
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.542
>> sec
>> [junit] Running org.apache.pig.test.TestParamSubPreproc
>> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.633
>> sec
>> [junit] Running org.apache.pig.test.TestParser
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.206
>> sec
>> [junit] Running org.apache.pig.test.TestPigContext
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.806
>> sec
>> [junit] Running org.apache.pig.test.TestPigFile
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 3.344
>> sec
>> [junit] Running org.apache.pig.test.TestPigScriptParser
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.441
>> sec
>> [junit] Running org.apache.pig.test.TestPigServer
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 5.76 sec
>> [junit] Running org.apache.pig.test.TestPigSplit
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.546
>> sec
>> [junit] Running org.apache.pig.test.TestReversibleLoadStore
>> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.622
>> sec
>> [junit] Running org.apache.pig.test.TestStore
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.139
>> sec
>> [junit] Running org.apache.pig.test.TestStreaming
>> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 1.074
>> sec
>>
>> test-mapreduce:
>>
>> testOneMode:
>> [mkdir] Created dir:
>> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
>> [junit] Running org.apache.pig.test.TestAlgebraicEval
>> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 74.738
>> sec
>> [junit] Running org.apache.pig.test.TestBinaryStorage
>> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 36.99
>> sec
>> [junit] Running org.apache.pig.test.TestBuiltin
>> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.513
>> sec
>> [junit] Running org.apache.pig.test.TestCmdLineParser
>> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.034
>> sec
>> [junit] Running org.apache.pig.test.TestCombiner
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 24.9 sec
>> [junit] Running org.apache.pig.test.TestCompressedFiles
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 29.658
>> sec
>> [junit] Running org.apache.pig.test.TestCustomSlicer
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 90.949
>> sec
>> [junit] Running org.apache.pig.test.TestDataBag
>> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.605
>> sec
>> [junit] Running org.apache.pig.test.TestDataModel
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.194
>> sec
>> [junit] Running org.apache.pig.test.TestEvalPipeline
>> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 203.826
>> sec
>> [junit] Running org.apache.pig.test.TestExGenCogroup
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 15.116
>> sec
>> [junit] Running org.apache.pig.test.TestExGenEval
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.609
>> sec
>> [junit] Running org.apache.pig.test.TestFilterOpNumeric
>> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 104.898
>> sec
>> [junit] Running org.apache.pig.test.TestFilterOpString
>> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 83.079
>> sec
>> [junit] Running org.apache.pig.test.TestGrunt
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.296
>> sec
>> [junit] Running org.apache.pig.test.TestInfixArithmetic
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 50.802
>> sec
>> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
>> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.328
>> sec
>> [junit] Running org.apache.pig.test.TestMapReduce
>> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 52.169
>> sec
>> [junit] Running org.apache.pig.test.TestMapReduceResultRecycling
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 43.827
>> sec
>> [junit] Running org.apache.pig.test.TestParamSubPreproc
>> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.724
>> sec
>> [junit] Running org.apache.pig.test.TestParser
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.08 sec
>> [junit] Running org.apache.pig.test.TestPigContext
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.209
>> sec
>> [junit] Running org.apache.pig.test.TestPigFile
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.95 sec
>> [junit] Running org.apache.pig.test.TestPigScriptParser
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.26 sec
>> [junit] Running org.apache.pig.test.TestPigServer
>> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 10.313
>> sec
>> [junit] Running org.apache.pig.test.TestPigSplit
>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 109.971
>> sec
>> [junit] Running org.apache.pig.test.TestReversibleLoadStore
>> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 11.803
>> sec
>> [junit] Running org.apache.pig.test.TestStore
>> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 125.27
>> sec
>> [junit] Running org.apache.pig.test.TestStreaming
>> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 142.835
>> sec
>>
>> test:
>>
>> BUILD SUCCESSFUL
>> Total time: 26 minutes 4 seconds
>> ucdev3:~/src/pig-apache3/trunk> cd
>> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
Re: Slow unit testing
Posted by Alan Gates <ga...@yahoo-inc.com>.
I run the unit tests on a mac laptop and am seeing similar time results.
Alan.
Olga Natkovich wrote:
> Pi,
>
> Inline my stdout content. Attached test output files. I ran this against
> latest trunk on a linux box.
>
> test-local:
>
> testOneMode:
> [mkdir] Created dir:
> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
> [junit] Running org.apache.pig.test.TestAlgebraicEval
> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.93 sec
> [junit] Running org.apache.pig.test.TestBinaryStorage
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.662
> sec
> [junit] Running org.apache.pig.test.TestBuiltin
> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 3.849
> sec
> [junit] Running org.apache.pig.test.TestCmdLineParser
> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.011
> sec
> [junit] Running org.apache.pig.test.TestCombiner
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.421
> sec
> [junit] Running org.apache.pig.test.TestCompressedFiles
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.337
> sec
> [junit] Running org.apache.pig.test.TestCustomSlicer
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.214
> sec
> [junit] Running org.apache.pig.test.TestDataBag
> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.461
> sec
> [junit] Running org.apache.pig.test.TestDataModel
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.126
> sec
> [junit] Running org.apache.pig.test.TestEvalPipeline
> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 5.035
> sec
> [junit] Running org.apache.pig.test.TestExGenCogroup
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.246
> sec
> [junit] Running org.apache.pig.test.TestExGenEval
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.172
> sec
> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 1.76 sec
> [junit] Running org.apache.pig.test.TestFilterOpString
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.385
> sec
> [junit] Running org.apache.pig.test.TestGrunt
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.238
> sec
> [junit] Running org.apache.pig.test.TestInfixArithmetic
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 57.608
> sec
> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.04
> sec
> [junit] Running org.apache.pig.test.TestMapReduce
> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 73.303
> sec
> [junit] Running org.apache.pig.test.TestMapReduceResultRecycling
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.542
> sec
> [junit] Running org.apache.pig.test.TestParamSubPreproc
> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.633
> sec
> [junit] Running org.apache.pig.test.TestParser
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.206
> sec
> [junit] Running org.apache.pig.test.TestPigContext
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.806
> sec
> [junit] Running org.apache.pig.test.TestPigFile
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 3.344
> sec
> [junit] Running org.apache.pig.test.TestPigScriptParser
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.441
> sec
> [junit] Running org.apache.pig.test.TestPigServer
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 5.76 sec
> [junit] Running org.apache.pig.test.TestPigSplit
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.546
> sec
> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.622
> sec
> [junit] Running org.apache.pig.test.TestStore
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.139
> sec
> [junit] Running org.apache.pig.test.TestStreaming
> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 1.074
> sec
>
> test-mapreduce:
>
> testOneMode:
> [mkdir] Created dir:
> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
> [junit] Running org.apache.pig.test.TestAlgebraicEval
> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 74.738
> sec
> [junit] Running org.apache.pig.test.TestBinaryStorage
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 36.99
> sec
> [junit] Running org.apache.pig.test.TestBuiltin
> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.513
> sec
> [junit] Running org.apache.pig.test.TestCmdLineParser
> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.034
> sec
> [junit] Running org.apache.pig.test.TestCombiner
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 24.9 sec
> [junit] Running org.apache.pig.test.TestCompressedFiles
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 29.658
> sec
> [junit] Running org.apache.pig.test.TestCustomSlicer
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 90.949
> sec
> [junit] Running org.apache.pig.test.TestDataBag
> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.605
> sec
> [junit] Running org.apache.pig.test.TestDataModel
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.194
> sec
> [junit] Running org.apache.pig.test.TestEvalPipeline
> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 203.826
> sec
> [junit] Running org.apache.pig.test.TestExGenCogroup
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 15.116
> sec
> [junit] Running org.apache.pig.test.TestExGenEval
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.609
> sec
> [junit] Running org.apache.pig.test.TestFilterOpNumeric
> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 104.898
> sec
> [junit] Running org.apache.pig.test.TestFilterOpString
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 83.079
> sec
> [junit] Running org.apache.pig.test.TestGrunt
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.296
> sec
> [junit] Running org.apache.pig.test.TestInfixArithmetic
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 50.802
> sec
> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.328
> sec
> [junit] Running org.apache.pig.test.TestMapReduce
> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 52.169
> sec
> [junit] Running org.apache.pig.test.TestMapReduceResultRecycling
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 43.827
> sec
> [junit] Running org.apache.pig.test.TestParamSubPreproc
> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.724
> sec
> [junit] Running org.apache.pig.test.TestParser
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.08 sec
> [junit] Running org.apache.pig.test.TestPigContext
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.209
> sec
> [junit] Running org.apache.pig.test.TestPigFile
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.95 sec
> [junit] Running org.apache.pig.test.TestPigScriptParser
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.26 sec
> [junit] Running org.apache.pig.test.TestPigServer
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 10.313
> sec
> [junit] Running org.apache.pig.test.TestPigSplit
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 109.971
> sec
> [junit] Running org.apache.pig.test.TestReversibleLoadStore
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 11.803
> sec
> [junit] Running org.apache.pig.test.TestStore
> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 125.27
> sec
> [junit] Running org.apache.pig.test.TestStreaming
> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 142.835
> sec
>
> test:
>
> BUILD SUCCESSFUL
> Total time: 26 minutes 4 seconds
> ucdev3:~/src/pig-apache3/trunk> cd
> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
Pi,
Inline my stdout content. Attached test output files. I ran this against
latest trunk on a linux box.
test-local:
testOneMode:
[mkdir] Created dir:
/home/olgan/src/pig-apache3/trunk/build/test/logs-local
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.93 sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.662
sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 3.849
sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.011
sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.421
sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.337
sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.214
sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.461
sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.126
sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 5.035
sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.246
sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.172
sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 1.76 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.385
sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.238
sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 57.608
sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.04
sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 73.303
sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.542
sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.633
sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.206
sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.806
sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 3.344
sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.441
sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 5.76 sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.546
sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.622
sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.139
sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 1.074
sec
test-mapreduce:
testOneMode:
[mkdir] Created dir:
/home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 74.738
sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 36.99
sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.513
sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.034
sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 24.9 sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 29.658
sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 90.949
sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.605
sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.194
sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 203.826
sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 15.116
sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.609
sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 104.898
sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 83.079
sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.296
sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 50.802
sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.328
sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 52.169
sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 43.827
sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.724
sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.08 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.209
sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.95 sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.26 sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 10.313
sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 109.971
sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 11.803
sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 125.27
sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 142.835
sec
test:
BUILD SUCCESSFUL
Total time: 26 minutes 4 seconds
ucdev3:~/src/pig-apache3/trunk> cd
~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
Below is stat from my latest test run which took 83 minutes. Is there anyone
else having the same problem?
Pi
test-local:
testOneMode:
[mkdir] Created dir: /root/IdeaProjects/Pig/build/test/logs-local
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 1.523 sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.507 sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.414 sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.047 sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.486 sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.257 sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.208 sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.303 sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.201 sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 8.903 sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.09 sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.579 sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 3.581 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.677 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.553 sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 53.791 sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.17 sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 57.749 sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.428 sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.317 sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.213 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.938 sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.501 sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.283 sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.338 sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.477 sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.661 sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.196 sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 1.187 sec
test-mapreduce:
testOneMode:
[mkdir] Created dir: /root/IdeaProjects/Pig/build/test/logs-mapreduce
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 258.721 sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 184.114 sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.538 sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.07 sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 173.987 sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 174.538 sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 236.571 sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.478 sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.165 sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 360.121 sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 160.593 sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 158.003 sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 262.014 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 234.839 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.159 sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 53.881 sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.396 sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 54.656 sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 198.565 sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.305 sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 158.771 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.762 sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 3.69 sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.336 sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 158.031 sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 258.747 sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 160.657 sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 280.73 sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 286.489 sec
test:
BUILD SUCCESSFUL
Total time: 83 minutes 8 seconds
On Tue, May 13, 2008 at 12:40 AM, pi song <pi...@gmail.com> wrote:
> When you run your tests next time, could you please zip all logs and send
> to me?
> This has been really annoying. I'll have to fix it.
>
> Pi
>
>
> On Tue, May 13, 2008 at 12:36 AM, Olga Natkovich <ol...@yahoo-inc.com>
> wrote:
>
> > I also see a lot of "noise" in my logs. I don't remember seeing this
> > before though it did not see to effect speed of my tests. I wonder if
> > this is somehow related to the test changes that happened recently.
> >
> >
> > ________________________________
> >
> > From: pi song [mailto:pi.songs@gmail.com]
> > Sent: Monday, May 12, 2008 7:30 AM
> > To: pig-dev@incubator.apache.org
> > Subject: Re: Slow unit testing
> >
> >
> > After looking at the log files:-
> > 1) It is not between tests but within the tests. Just my screen
> > is updating too slowly.
> > 2) My log files look very messy. Is your file as messy as mine?
> >
> > Pi
> >
> >
> > On Tue, May 13, 2008 at 12:20 AM, pi song <pi...@gmail.com>
> > wrote:
> >
> >
> > This started to happen since the introduction of
> > Hadoop16 (which includes changes in HOD). The time shown in ANT-Junit is
> > not accurate. What I can see here is that time spent doing nothing
> > between each tests is long but not taken into account.
> >
> > Is it possible to be something to do with connecting to
> > HOD?
> >
> > [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> > elapsed: 262.014 sec
> > [junit] Running
> > org.apache.pig.test.TestFilterOpString
> > [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> > elapsed: 234.839 sec
> > [junit] Running org.apache.pig.test.TestGrunt
> > [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.159 sec
> >
> > Pi
> >
> >
> > On Tue, May 13, 2008 at 12:15 AM, pi song
> > <pi...@gmail.com> wrote:
> >
> >
> > Already more than 40 minutes.
> >
> >
> > On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich
> > <ol...@yahoo-inc.com> wrote:
> >
> >
> > I run the tests on the regular basis and
> > don't see a proble. MR tests
> > take about 20 minites for me. How long
> > does it take for you?
> >
> > Olga
> >
> >
> > > -----Original Message-----
> > > From: pi song
> > [mailto:pi.songs@gmail.com]
> > > Sent: Monday, May 12, 2008 6:57 AM
> > > To: pig-dev@incubator.apache.org
> > > Subject: Slow unit testing
> > >
> > > It has been a while since we moved
> > from Hadoop15 to Hadoop16
> > > that all the MapReduce unit tests run
> > significantly slower
> > > than before (at least 2-3x).
> > > What is the reason behind this? Is
> > there a way to fix?
> > >
> > > Since everyone has to run at least a
> > full test before
> > > submitting a patch.
> > > This is not productive from
> > developer's point of view.
> > >
> > > Pi
> > >
> >
> >
> >
> >
> >
> >
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
When you run your tests next time, could you please zip all logs and send to
me?
This has been really annoying. I'll have to fix it.
Pi
On Tue, May 13, 2008 at 12:36 AM, Olga Natkovich <ol...@yahoo-inc.com>
wrote:
> I also see a lot of "noise" in my logs. I don't remember seeing this
> before though it did not see to effect speed of my tests. I wonder if
> this is somehow related to the test changes that happened recently.
>
>
> ________________________________
>
> From: pi song [mailto:pi.songs@gmail.com]
> Sent: Monday, May 12, 2008 7:30 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Slow unit testing
>
>
> After looking at the log files:-
> 1) It is not between tests but within the tests. Just my screen
> is updating too slowly.
> 2) My log files look very messy. Is your file as messy as mine?
>
> Pi
>
>
> On Tue, May 13, 2008 at 12:20 AM, pi song <pi...@gmail.com>
> wrote:
>
>
> This started to happen since the introduction of
> Hadoop16 (which includes changes in HOD). The time shown in ANT-Junit is
> not accurate. What I can see here is that time spent doing nothing
> between each tests is long but not taken into account.
>
> Is it possible to be something to do with connecting to
> HOD?
>
> [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> elapsed: 262.014 sec
> [junit] Running
> org.apache.pig.test.TestFilterOpString
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> elapsed: 234.839 sec
> [junit] Running org.apache.pig.test.TestGrunt
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> elapsed: 0.159 sec
>
> Pi
>
>
> On Tue, May 13, 2008 at 12:15 AM, pi song
> <pi...@gmail.com> wrote:
>
>
> Already more than 40 minutes.
>
>
> On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich
> <ol...@yahoo-inc.com> wrote:
>
>
> I run the tests on the regular basis and
> don't see a proble. MR tests
> take about 20 minites for me. How long
> does it take for you?
>
> Olga
>
>
> > -----Original Message-----
> > From: pi song
> [mailto:pi.songs@gmail.com]
> > Sent: Monday, May 12, 2008 6:57 AM
> > To: pig-dev@incubator.apache.org
> > Subject: Slow unit testing
> >
> > It has been a while since we moved
> from Hadoop15 to Hadoop16
> > that all the MapReduce unit tests run
> significantly slower
> > than before (at least 2-3x).
> > What is the reason behind this? Is
> there a way to fix?
> >
> > Since everyone has to run at least a
> full test before
> > submitting a patch.
> > This is not productive from
> developer's point of view.
> >
> > Pi
> >
>
>
>
>
>
>
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
I also see a lot of "noise" in my logs. I don't remember seeing this
before though it did not see to effect speed of my tests. I wonder if
this is somehow related to the test changes that happened recently.
________________________________
From: pi song [mailto:pi.songs@gmail.com]
Sent: Monday, May 12, 2008 7:30 AM
To: pig-dev@incubator.apache.org
Subject: Re: Slow unit testing
After looking at the log files:-
1) It is not between tests but within the tests. Just my screen
is updating too slowly.
2) My log files look very messy. Is your file as messy as mine?
Pi
On Tue, May 13, 2008 at 12:20 AM, pi song <pi...@gmail.com>
wrote:
This started to happen since the introduction of
Hadoop16 (which includes changes in HOD). The time shown in ANT-Junit is
not accurate. What I can see here is that time spent doing nothing
between each tests is long but not taken into account.
Is it possible to be something to do with connecting to
HOD?
[junit] Tests run: 8, Failures: 0, Errors: 0, Time
elapsed: 262.014 sec
[junit] Running
org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time
elapsed: 234.839 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time
elapsed: 0.159 sec
Pi
On Tue, May 13, 2008 at 12:15 AM, pi song
<pi...@gmail.com> wrote:
Already more than 40 minutes.
On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich
<ol...@yahoo-inc.com> wrote:
I run the tests on the regular basis and
don't see a proble. MR tests
take about 20 minites for me. How long
does it take for you?
Olga
> -----Original Message-----
> From: pi song
[mailto:pi.songs@gmail.com]
> Sent: Monday, May 12, 2008 6:57 AM
> To: pig-dev@incubator.apache.org
> Subject: Slow unit testing
>
> It has been a while since we moved
from Hadoop15 to Hadoop16
> that all the MapReduce unit tests run
significantly slower
> than before (at least 2-3x).
> What is the reason behind this? Is
there a way to fix?
>
> Since everyone has to run at least a
full test before
> submitting a patch.
> This is not productive from
developer's point of view.
>
> Pi
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
After looking at the log files:-
1) It is not between tests but within the tests. Just my screen is updating
too slowly.
2) My log files look very messy. Is your file as messy as mine?
Pi
On Tue, May 13, 2008 at 12:20 AM, pi song <pi...@gmail.com> wrote:
> This started to happen since the introduction of Hadoop16 (which includes
> changes in HOD). The time shown in ANT-Junit is not accurate. What I can see
> here is that time spent doing nothing between each tests is long but not
> taken into account.
>
> Is it possible to be something to do with connecting to HOD?
>
> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 262.014
> sec
> [junit] Running org.apache.pig.test.TestFilterOpString
> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 234.839
> sec
> [junit] Running org.apache.pig.test.TestGrunt
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.159 sec
>
> Pi
>
>
> On Tue, May 13, 2008 at 12:15 AM, pi song <pi...@gmail.com> wrote:
>
> > Already more than 40 minutes.
> >
> >
> > On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich <ol...@yahoo-inc.com>
> > wrote:
> >
> > > I run the tests on the regular basis and don't see a proble. MR tests
> > > take about 20 minites for me. How long does it take for you?
> > >
> > > Olga
> > >
> > > > -----Original Message-----
> > > > From: pi song [mailto:pi.songs@gmail.com]
> > > > Sent: Monday, May 12, 2008 6:57 AM
> > > > To: pig-dev@incubator.apache.org
> > > > Subject: Slow unit testing
> > > >
> > > > It has been a while since we moved from Hadoop15 to Hadoop16
> > > > that all the MapReduce unit tests run significantly slower
> > > > than before (at least 2-3x).
> > > > What is the reason behind this? Is there a way to fix?
> > > >
> > > > Since everyone has to run at least a full test before
> > > > submitting a patch.
> > > > This is not productive from developer's point of view.
> > > >
> > > > Pi
> > > >
> > >
> >
> >
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
This started to happen since the introduction of Hadoop16 (which includes
changes in HOD). The time shown in ANT-Junit is not accurate. What I can see
here is that time spent doing nothing between each tests is long but not
taken into account.
Is it possible to be something to do with connecting to HOD?
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 262.014 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 234.839 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.159 sec
Pi
On Tue, May 13, 2008 at 12:15 AM, pi song <pi...@gmail.com> wrote:
> Already more than 40 minutes.
>
>
> On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich <ol...@yahoo-inc.com>
> wrote:
>
> > I run the tests on the regular basis and don't see a proble. MR tests
> > take about 20 minites for me. How long does it take for you?
> >
> > Olga
> >
> > > -----Original Message-----
> > > From: pi song [mailto:pi.songs@gmail.com]
> > > Sent: Monday, May 12, 2008 6:57 AM
> > > To: pig-dev@incubator.apache.org
> > > Subject: Slow unit testing
> > >
> > > It has been a while since we moved from Hadoop15 to Hadoop16
> > > that all the MapReduce unit tests run significantly slower
> > > than before (at least 2-3x).
> > > What is the reason behind this? Is there a way to fix?
> > >
> > > Since everyone has to run at least a full test before
> > > submitting a patch.
> > > This is not productive from developer's point of view.
> > >
> > > Pi
> > >
> >
>
>
Re: Slow unit testing
Posted by pi song <pi...@gmail.com>.
Already more than 40 minutes.
On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich <ol...@yahoo-inc.com>
wrote:
> I run the tests on the regular basis and don't see a proble. MR tests
> take about 20 minites for me. How long does it take for you?
>
> Olga
>
> > -----Original Message-----
> > From: pi song [mailto:pi.songs@gmail.com]
> > Sent: Monday, May 12, 2008 6:57 AM
> > To: pig-dev@incubator.apache.org
> > Subject: Slow unit testing
> >
> > It has been a while since we moved from Hadoop15 to Hadoop16
> > that all the MapReduce unit tests run significantly slower
> > than before (at least 2-3x).
> > What is the reason behind this? Is there a way to fix?
> >
> > Since everyone has to run at least a full test before
> > submitting a patch.
> > This is not productive from developer's point of view.
> >
> > Pi
> >
>
RE: Slow unit testing
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
I run the tests on the regular basis and don't see a proble. MR tests
take about 20 minites for me. How long does it take for you?
Olga
> -----Original Message-----
> From: pi song [mailto:pi.songs@gmail.com]
> Sent: Monday, May 12, 2008 6:57 AM
> To: pig-dev@incubator.apache.org
> Subject: Slow unit testing
>
> It has been a while since we moved from Hadoop15 to Hadoop16
> that all the MapReduce unit tests run significantly slower
> than before (at least 2-3x).
> What is the reason behind this? Is there a way to fix?
>
> Since everyone has to run at least a full test before
> submitting a patch.
> This is not productive from developer's point of view.
>
> Pi
>