You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Matt Tanquary <ma...@gmail.com> on 2010/10/02 00:07:18 UTC
Local compiles and testing
I am trying to make some changes to the source for local testing. Can
you suggest what I need to do. I am able to compile successfully, but
running mahout after recompile seems to have no effect.
Thanks,
-M@
Re: Local compiles and testing
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
No, it doesn't manage svn for you, only the maven repository downloads
get done. Run mvn clean install -DskipTests=true to get a clean set of
jar/job files suitable for CLI invocation any time you make changes in
Eclipse. They won't get overwritten.
On 10/2/10 3:41 PM, Matt Tanquary wrote:
> If I run mvn clean install, will it build with my changes? I assumed
> that would pull code from apache servers. Again...ignorance...:)
>
> On Fri, Oct 1, 2010 at 3:56 PM, Jeff Eastman<jd...@windwardsolutions.com> wrote:
>> Did you run mvn clean install from the command line? Compiling the files in
>> Eclipse is not sufficient to update the job files.
>>
>>
>> On 10/1/10 6:44 PM, Matt Tanquary wrote:
>>> I loaded the projects to Eclipse via Maven.
>>> I updated KMeansDriver line 414 to change the "Running Clustering"
>>> text to something that I would recognize as a change.
>>> I forced rebuild of the projects after changing
>>> org.apache.mahout.clustering.kmeans.KMeansDriver
>>> I noticed that the
>>> core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
>>> file date stamp was updated
>>> I attempted to run mahout kmeans and didn't get the expected log.info
>>> text, it was still the original text.
>>>
>>> I see that these .job files are used, although I'm not really sure how
>>> they work as opposed to .jar files, and I'm sure my ignorance is
>>> getting in my way. :)
>>>
>>> -M@
>>>
>>> On Fri, Oct 1, 2010 at 3:23 PM, Ted Dunning<te...@gmail.com> wrote:
>>>> Can you provide a transcript of the commands you use to do this?
>>>>
>>>> You might even try computing an md5sum on all of the source files in the
>>>> src
>>>> directory and the class files in the
>>>> target directory to verify that you know exactly what is changing.
>>>>
>>>> In general, when I have these kinds of problems, it is because I am
>>>> erroneously assuming something. That makes
>>>> it very hard for me to tell somebody else what my problem is because when
>>>> I
>>>> describe the situation I am not actually
>>>> describing the situation as it is.
>>>>
>>>> On Fri, Oct 1, 2010 at 3:07 PM, Matt
>>>> Tanquary<ma...@gmail.com>wrote:
>>>>
>>>>> I am trying to make some changes to the source for local testing. Can
>>>>> you suggest what I need to do. I am able to compile successfully, but
>>>>> running mahout after recompile seems to have no effect.
>>>>>
>>>>> Thanks,
>>>>> -M@
>>>>>
>>>
>>
>
>
Re: Local compiles and testing
Posted by Matt Tanquary <ma...@gmail.com>.
If I run mvn clean install, will it build with my changes? I assumed
that would pull code from apache servers. Again...ignorance...:)
On Fri, Oct 1, 2010 at 3:56 PM, Jeff Eastman <jd...@windwardsolutions.com> wrote:
> Did you run mvn clean install from the command line? Compiling the files in
> Eclipse is not sufficient to update the job files.
>
>
> On 10/1/10 6:44 PM, Matt Tanquary wrote:
>>
>> I loaded the projects to Eclipse via Maven.
>> I updated KMeansDriver line 414 to change the "Running Clustering"
>> text to something that I would recognize as a change.
>> I forced rebuild of the projects after changing
>> org.apache.mahout.clustering.kmeans.KMeansDriver
>> I noticed that the
>> core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
>> file date stamp was updated
>> I attempted to run mahout kmeans and didn't get the expected log.info
>> text, it was still the original text.
>>
>> I see that these .job files are used, although I'm not really sure how
>> they work as opposed to .jar files, and I'm sure my ignorance is
>> getting in my way. :)
>>
>> -M@
>>
>> On Fri, Oct 1, 2010 at 3:23 PM, Ted Dunning<te...@gmail.com> wrote:
>>>
>>> Can you provide a transcript of the commands you use to do this?
>>>
>>> You might even try computing an md5sum on all of the source files in the
>>> src
>>> directory and the class files in the
>>> target directory to verify that you know exactly what is changing.
>>>
>>> In general, when I have these kinds of problems, it is because I am
>>> erroneously assuming something. That makes
>>> it very hard for me to tell somebody else what my problem is because when
>>> I
>>> describe the situation I am not actually
>>> describing the situation as it is.
>>>
>>> On Fri, Oct 1, 2010 at 3:07 PM, Matt
>>> Tanquary<ma...@gmail.com>wrote:
>>>
>>>> I am trying to make some changes to the source for local testing. Can
>>>> you suggest what I need to do. I am able to compile successfully, but
>>>> running mahout after recompile seems to have no effect.
>>>>
>>>> Thanks,
>>>> -M@
>>>>
>>
>>
>
>
--
Have you thanked a teacher today? ---> http://www.liftateacher.org
Re: Local compiles and testing
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
Did you run mvn clean install from the command line? Compiling the
files in Eclipse is not sufficient to update the job files.
On 10/1/10 6:44 PM, Matt Tanquary wrote:
> I loaded the projects to Eclipse via Maven.
> I updated KMeansDriver line 414 to change the "Running Clustering"
> text to something that I would recognize as a change.
> I forced rebuild of the projects after changing
> org.apache.mahout.clustering.kmeans.KMeansDriver
> I noticed that the
> core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
> file date stamp was updated
> I attempted to run mahout kmeans and didn't get the expected log.info
> text, it was still the original text.
>
> I see that these .job files are used, although I'm not really sure how
> they work as opposed to .jar files, and I'm sure my ignorance is
> getting in my way. :)
>
> -M@
>
> On Fri, Oct 1, 2010 at 3:23 PM, Ted Dunning<te...@gmail.com> wrote:
>> Can you provide a transcript of the commands you use to do this?
>>
>> You might even try computing an md5sum on all of the source files in the src
>> directory and the class files in the
>> target directory to verify that you know exactly what is changing.
>>
>> In general, when I have these kinds of problems, it is because I am
>> erroneously assuming something. That makes
>> it very hard for me to tell somebody else what my problem is because when I
>> describe the situation I am not actually
>> describing the situation as it is.
>>
>> On Fri, Oct 1, 2010 at 3:07 PM, Matt Tanquary<ma...@gmail.com>wrote:
>>
>>> I am trying to make some changes to the source for local testing. Can
>>> you suggest what I need to do. I am able to compile successfully, but
>>> running mahout after recompile seems to have no effect.
>>>
>>> Thanks,
>>> -M@
>>>
>
>
Re: Local compiles and testing
Posted by Ted Dunning <te...@gmail.com>.
To rebuild the job jar use maven's command [mvn -DskipTests install] (but
make sure you run the tests occasionally)
You can't trust Eclipse to understand the entire build. It will be ok if
you are running unit tests, but if you try to submit
a Hadoop job, you need to package everything up.
On Sat, Oct 2, 2010 at 12:39 PM, Matt Tanquary <ma...@gmail.com>wrote:
> > Did you just recompile? Or did you rebuild the job and/or jar files?
> >
> I used Eclipse "Clean" and "Rebuild All" to recompile. I just
> recompiled, but I did re-jar the core jar file manually.
>
Re: Local compiles and testing
Posted by Matt Tanquary <ma...@gmail.com>.
On Fri, Oct 1, 2010 at 3:53 PM, Ted Dunning <te...@gmail.com> wrote:
> Matt,
>
> This is good detail.
>
> On Fri, Oct 1, 2010 at 3:44 PM, Matt Tanquary <ma...@gmail.com>wrote:
>
>> I forced rebuild of the projects after changing
>> org.apache.mahout.clustering.kmeans.KMeansDriver
>> I noticed that the
>> core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
>> file date stamp was updated
>>
>
> How did you force the rebuild? Unless you are running a sequential
> algorithm, you
> probably need to build the jar file which may not be done by the IDE's
> normal recompile
> step.
>
> Did you just recompile? Or did you rebuild the job and/or jar files?
>
I used Eclipse "Clean" and "Rebuild All" to recompile. I just
recompiled, but I did re-jar the core jar file manually.
> I attempted to run mahout kmeans and didn't get the expected log.info
>> text, it was still the original text.
>>
>
> How did you run the program? Directly from within the IDE?
>
I ran the program using the mahout script from the bash shell using
the mahout script: mahout kmeans ...
> I see that these .job files are used, although I'm not really sure how
>> they work as opposed to .jar files,
>>
>
> the job files *are* jar files, but they are packaged up ready to run with
> Hadoop.
>
>
>> and I'm sure my ignorance is
>> getting in my way. :)
>>
>
> Join the club! My ignorance gets in my way all the time.
>
Thanks for your help. :-)
--
Have you thanked a teacher today? ---> http://www.liftateacher.org
Re: Local compiles and testing
Posted by Ted Dunning <te...@gmail.com>.
Matt,
This is good detail.
On Fri, Oct 1, 2010 at 3:44 PM, Matt Tanquary <ma...@gmail.com>wrote:
> I forced rebuild of the projects after changing
> org.apache.mahout.clustering.kmeans.KMeansDriver
> I noticed that the
> core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
> file date stamp was updated
>
How did you force the rebuild? Unless you are running a sequential
algorithm, you
probably need to build the jar file which may not be done by the IDE's
normal recompile
step.
Did you just recompile? Or did you rebuild the job and/or jar files?
I attempted to run mahout kmeans and didn't get the expected log.info
> text, it was still the original text.
>
How did you run the program? Directly from within the IDE?
I see that these .job files are used, although I'm not really sure how
> they work as opposed to .jar files,
>
the job files *are* jar files, but they are packaged up ready to run with
Hadoop.
> and I'm sure my ignorance is
> getting in my way. :)
>
Join the club! My ignorance gets in my way all the time.
Re: Local compiles and testing
Posted by Matt Tanquary <ma...@gmail.com>.
I loaded the projects to Eclipse via Maven.
I updated KMeansDriver line 414 to change the "Running Clustering"
text to something that I would recognize as a change.
I forced rebuild of the projects after changing
org.apache.mahout.clustering.kmeans.KMeansDriver
I noticed that the
core/target/classes/org/apache/mahout/clustering/kmeans/KMeansDriver.class
file date stamp was updated
I attempted to run mahout kmeans and didn't get the expected log.info
text, it was still the original text.
I see that these .job files are used, although I'm not really sure how
they work as opposed to .jar files, and I'm sure my ignorance is
getting in my way. :)
-M@
On Fri, Oct 1, 2010 at 3:23 PM, Ted Dunning <te...@gmail.com> wrote:
> Can you provide a transcript of the commands you use to do this?
>
> You might even try computing an md5sum on all of the source files in the src
> directory and the class files in the
> target directory to verify that you know exactly what is changing.
>
> In general, when I have these kinds of problems, it is because I am
> erroneously assuming something. That makes
> it very hard for me to tell somebody else what my problem is because when I
> describe the situation I am not actually
> describing the situation as it is.
>
> On Fri, Oct 1, 2010 at 3:07 PM, Matt Tanquary <ma...@gmail.com>wrote:
>
>> I am trying to make some changes to the source for local testing. Can
>> you suggest what I need to do. I am able to compile successfully, but
>> running mahout after recompile seems to have no effect.
>>
>> Thanks,
>> -M@
>>
>
--
Have you thanked a teacher today? ---> http://www.liftateacher.org
Re: Local compiles and testing
Posted by Ted Dunning <te...@gmail.com>.
Can you provide a transcript of the commands you use to do this?
You might even try computing an md5sum on all of the source files in the src
directory and the class files in the
target directory to verify that you know exactly what is changing.
In general, when I have these kinds of problems, it is because I am
erroneously assuming something. That makes
it very hard for me to tell somebody else what my problem is because when I
describe the situation I am not actually
describing the situation as it is.
On Fri, Oct 1, 2010 at 3:07 PM, Matt Tanquary <ma...@gmail.com>wrote:
> I am trying to make some changes to the source for local testing. Can
> you suggest what I need to do. I am able to compile successfully, but
> running mahout after recompile seems to have no effect.
>
> Thanks,
> -M@
>