You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mahout.apache.org by "Drew Farris (JIRA)" <ji...@apache.org> on 2010/01/07 04:10:56 UTC

[jira] Created: (MAHOUT-238) Further Dependency Cleanup

Further Dependency Cleanup
--------------------------

Key: MAHOUT-238
URL: https://issues.apache.org/jira/browse/MAHOUT-238
Project: Mahout
Issue Type: Sub-task
Reporter: Drew Farris
Priority: Minor
Fix For: 0.3

Further dependency cleanup is required, mainly to set the right hadoop dependency for mahout-math and fix exclusions for the hadoop dependency in the parent pom. Other minor cleanups too.

The patch includes the following changes:

maven (parent pom)
* added inceptionYear (2008)
* removed some exclusions for hadoop dependency: avro, commons-codec, commons-httpclient in the dependendy management section.
* removed javax.mail dependency

mahout-math
* switched from o.a.m.hadoop:hadoop-core dependency to new o.a.hadoop:hadoop-core dependency used in core, version specified in dependencyManagement section of parent pom.
* removed unnecessary compile scope from gson dependency

mahout-core
* removed: kfs, jets3t, xmlenc, unused, originally added to support old o.a.mahout.hadoop:hadoop-core:0.20.1 dependency
* removed: commons-httpclient, now added transitively from new o.a.hadoop:hadoop-core:0.20.2-SNAPSHOT dependency
* set slf4j-jcl to test scope.
* removed: watchmaker-swing, added later in mahout-examples where it is actually used.
* fixed uncommons-maths groupId
* removed unused lucene-analyzers dependency.
* added easymock dependencies explicitly

mahout-utils
* removed unused easymock dependencies

mahout-examples
* added watchmaker-framework and watchmaker-swing

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Ted Dunning <te...@gmail.com>.

I think that 3.2.2 is the current release for ZK

On Thu, Jan 21, 2010 at 2:12 AM, zhao zhendong <zh...@gmail.com>wrote:

> 2) and add zookeeper-3.2.1.jar.




-- 
Ted Dunning, CTO
DeepDyve

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Stack <st...@duboce.net>.

We're working on hbase 0.21 as being the first hbase that shows up in
a maven repo.
St.Ack

On Fri, Jan 22, 2010 at 1:12 PM, Mahadev Konar <ma...@yahoo-inc.com> wrote:
> Unfortunately no.. We are planning to deploy 3.3 as the first version on
> maven repo.
>
>
> Thanks
> mahadev
>
>
> On 1/22/10 12:58 PM, "Ted Dunning" <te...@gmail.com> wrote:
>
>> Is ZK 3.2.2 in a maven repository somewhere?
>>
>> ---------- Forwarded message ----------
>> From: Drew Farris <dr...@gmail.com>
>> Date: Fri, Jan 22, 2010 at 11:47 AM
>> Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
>> To: mahout-dev@lucene.apache.org
>>
>>
>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>> maven repo at this point, so Mahout would have to roll and deploy
>> these. What was the process that was followed to build and deploy the
>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>> could submit a patch to Mahout for, or better left for the committers?
>>
>> As Ted pointed out, yes the release of zk is 3.2.2
>>
>> Drew
>>
>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
>> wrote:
>>> Hi Drew,
>>>
>>> I propose to
>>> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
>>> and hbased-platform is based on this version,
>>>
>>> 2) and add zookeeper-3.2.1.jar.
>>>
>>> Cheers,
>>> Zhendong
>>>
>>> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>>> wrote:
>>>
>>>> Hi Drew,
>>>>
>>>> Including a source code in snapshots that will be great.
>>>>
>>>> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>>>> it's not convenient for me to debug the code.
>>>>
>>>> Cheers,
>>>> Zhendong
>>>>
>>>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>>> wrote:
>>>>
>>>>> I wonder if we can get the hadoop people to include source jars with
>>>>> their snapshots?
>>>>>
>>>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>>>>> I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>>>>>> don't have to maintain our own version. The downside is relying on a
>>>>>> SNAPSHOT, but seems worth it to me.
>>>>>>
>>>>>> On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
>>>>> wrote:
>>>>>>> Thanks Drew,
>>>>>>>
>>>>>>> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>>>> reason is
>>>>>>> obvious :)
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Zhendong
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -------------------------------------------------------------
>>>>
>>>> Zhen-Dong Zhao (Maxim)
>>>>
>>>> <><<><><><><><><><>><><><><><>>>>>>
>>>>
>>>> Department of Computer Science
>>>> School of Computing
>>>> National University of Singapore
>>>>
>>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> -------------------------------------------------------------
>>>
>>> Zhen-Dong Zhao (Maxim)
>>>
>>> <><<><><><><><><><>><><><><><>>>>>>
>>>
>>> Department of Computer Science
>>> School of Computing
>>> National University of Singapore
>>>
>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>
>>
>>
>
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Mahadev Konar <ma...@yahoo-inc.com>.

Unfortunately no.. We are planning to deploy 3.3 as the first version on
maven repo.


Thanks
mahadev


On 1/22/10 12:58 PM, "Ted Dunning" <te...@gmail.com> wrote:

> Is ZK 3.2.2 in a maven repository somewhere?
> 
> ---------- Forwarded message ----------
> From: Drew Farris <dr...@gmail.com>
> Date: Fri, Jan 22, 2010 at 11:47 AM
> Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
> To: mahout-dev@lucene.apache.org
> 
> 
> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
> maven repo at this point, so Mahout would have to roll and deploy
> these. What was the process that was followed to build and deploy the
> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
> could submit a patch to Mahout for, or better left for the committers?
> 
> As Ted pointed out, yes the release of zk is 3.2.2
> 
> Drew
> 
> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
> wrote:
>> Hi Drew,
>> 
>> I propose to
>> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
>> and hbased-platform is based on this version,
>> 
>> 2) and add zookeeper-3.2.1.jar.
>> 
>> Cheers,
>> Zhendong
>> 
>> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>> wrote:
>> 
>>> Hi Drew,
>>> 
>>> Including a source code in snapshots that will be great.
>>> 
>>> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>>> it's not convenient for me to debug the code.
>>> 
>>> Cheers,
>>> Zhendong
>>> 
>>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>> wrote:
>>> 
>>>> I wonder if we can get the hadoop people to include source jars with
>>>> their snapshots?
>>>> 
>>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>>>> I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>>>>> don't have to maintain our own version. The downside is relying on a
>>>>> SNAPSHOT, but seems worth it to me.
>>>>> 
>>>>> On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
>>>> wrote:
>>>>>> Thanks Drew,
>>>>>> 
>>>>>> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>>> reason is
>>>>>> obvious :)
>>>>>> 
>>>>>> Cheers,
>>>>>> Zhendong
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> -------------------------------------------------------------
>>> 
>>> Zhen-Dong Zhao (Maxim)
>>> 
>>> <><<><><><><><><><>><><><><><>>>>>>
>>> 
>>> Department of Computer Science
>>> School of Computing
>>> National University of Singapore
>>> 
>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>> 
>>> 
>>> 
>> 
>> 
>> --
>> -------------------------------------------------------------
>> 
>> Zhen-Dong Zhao (Maxim)
>> 
>> <><<><><><><><><><>><><><><><>>>>>>
>> 
>> Department of Computer Science
>> School of Computing
>> National University of Singapore
>> 
>>>>>>>>> <><><><><><><><<><>><><<<<<<
>> 
> 
>

Fwd: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Ted Dunning <te...@gmail.com>.

Is ZK 3.2.2 in a maven repository somewhere?

---------- Forwarded message ----------
From: Drew Farris <dr...@gmail.com>
Date: Fri, Jan 22, 2010 at 11:47 AM
Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
To: mahout-dev@lucene.apache.org


Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
maven repo at this point, so Mahout would have to roll and deploy
these. What was the process that was followed to build and deploy the
mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
could submit a patch to Mahout for, or better left for the committers?

As Ted pointed out, yes the release of zk is 3.2.2

Drew

On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
wrote:
> Hi Drew,
>
> I propose to
> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
> and hbased-platform is based on this version,
>
> 2) and add zookeeper-3.2.1.jar.
>
> Cheers,
> Zhendong
>
> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>wrote:
>
>> Hi Drew,
>>
>> Including a source code in snapshots that will be great.
>>
>> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>> it's not convenient for me to debug the code.
>>
>> Cheers,
>> Zhendong
>>
>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>wrote:
>>
>>> I wonder if we can get the hadoop people to include source jars with
>>> their snapshots?
>>>
>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>>> > don't have to maintain our own version. The downside is relying on a
>>> > SNAPSHOT, but seems worth it to me.
>>> >
>>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
>>> wrote:
>>> >> Thanks Drew,
>>> >>
>>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>> reason is
>>> >> obvious :)
>>> >>
>>> >> Cheers,
>>> >> Zhendong
>>> >>
>>> >>
>>> >
>>>
>>
>>
>>
>> --
>> -------------------------------------------------------------
>>
>> Zhen-Dong Zhao (Maxim)
>>
>> <><<><><><><><><><>><><><><><>>>>>>
>>
>> Department of Computer Science
>> School of Computing
>> National University of Singapore
>>
>> >>>>>>><><><><><><><><<><>><><<<<<<
>>
>>
>>
>
>
> --
> -------------------------------------------------------------
>
> Zhen-Dong Zhao (Maxim)
>
> <><<><><><><><><><>><><><><><>>>>>>
>
> Department of Computer Science
> School of Computing
> National University of Singapore
>
>>>>>>>><><><><><><><><<><>><><<<<<<
>



-- 
Ted Dunning, CTO
DeepDyve

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Drew Farris <dr...@gmail.com>.

Great, I suspected someone within Mahout had deployed them, it is nice
to have the reference. Thanks Isabel. It looks like a committer will
need to take care of this.

I propose we deploy the following artifacts under the mahout repo:

A time-stamped snapshot of hadoop 0.20.2, possibly including
hand-build source and javadoc jars.
hbase-0.20.2
zookeeper 3.2.2

I do not prefer deploying non-release hand built items, but I think
this is the shortest route to meeting the original request. I looked a
bit at the hadoop build process and it will take some time for me to
come up to speed with ivy so that I can coax it into building source
and javadoc jars automatically, but I haven't given up on that
entirely.

I can put together a tarfile containing the hadoop artifacts and
provide the commands for someone to run to perform the sign and
deploy. I can include the release jars of hbase and zookeeper as well.

Thrift is a transitive dependency for hbase, I don't believe Mahout
needs to include it because we're building against hbase, not using it
to create a runtime environment.

Does this sound ok? Amongst the committers are there any volunteers
who could work with me to make this happen?

Drew

On Mon, Jan 25, 2010 at 9:42 AM, Isabel Drost <is...@apache.org> wrote:
> On Mon Grant Ingersoll <gs...@apache.org> wrote:
>
>> We put it up there.
>
> http://www.lucidimagination.com/search/document/621471200d2182bb/dependencies_outside_maven_central_was_oh_joy#621471200d2182bb
>
> is the link to the posting by Jukka explaining exactly how it was done.
>
> Isabel
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Isabel Drost <is...@apache.org>.

On Mon Grant Ingersoll <gs...@apache.org> wrote:

> We put it up there.

http://www.lucidimagination.com/search/document/621471200d2182bb/dependencies_outside_maven_central_was_oh_joy#621471200d2182bb

is the link to the posting by Jukka explaining exactly how it was done.

Isabel

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Grant Ingersoll <gs...@apache.org>.

We put it up there.

On Jan 23, 2010, at 11:25 AM, Drew Farris wrote:

> How was the current hbase deployed to the apache repo under the mahout
> package name? I'd be happy to help get these (and a newer hadoop)
> rolled in, I'm just unsure of the process.
> 
> On Fri, Jan 22, 2010 at 5:07 PM, Ted Dunning <te...@gmail.com> wrote:
>> Feedback from ZK and hbase folks:
>> 
>> Mahadev Konar said:
>> Unfortunately no.. We are planning to deploy 3.3 as the first version on
>> maven repo.
>> 
>> StAck said:
>> We're working on hbase 0.21 as being the first hbase that shows up in
>> a maven repo.
>> 
>> It looks like we are on our own.
>> 
>> On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris <dr...@gmail.com> wrote:
>> 
>>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>>> maven repo at this point, so Mahout would have to roll and deploy
>>> these. What was the process that was followed to build and deploy the
>>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>>> could submit a patch to Mahout for, or better left for the committers?
>>> 
>>> As Ted pointed out, yes the release of zk is 3.2.2
>>> 
>>> Drew
>>> 
>>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
>>> wrote:
>>>> Hi Drew,
>>>> 
>>>> I propose to
>>>> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
>>>> and hbased-platform is based on this version,
>>>> 
>>>> 2) and add zookeeper-3.2.1.jar.
>>>> 
>>>> Cheers,
>>>> Zhendong
>>>> 
>>>> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>>>> wrote:
>>>> 
>>>>> Hi Drew,
>>>>> 
>>>>> Including a source code in snapshots that will be great.
>>>>> 
>>>>> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>>>>> it's not convenient for me to debug the code.
>>>>> 
>>>>> Cheers,
>>>>> Zhendong
>>>>> 
>>>>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>>>> wrote:
>>>>> 
>>>>>> I wonder if we can get the hadoop people to include source jars with
>>>>>> their snapshots?
>>>>>> 
>>>>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>>>>>> I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>>>>>>> don't have to maintain our own version. The downside is relying on a
>>>>>>> SNAPSHOT, but seems worth it to me.
>>>>>>> 
>>>>>>> On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <
>>> zhaozhendong@gmail.com>
>>>>>> wrote:
>>>>>>>> Thanks Drew,
>>>>>>>> 
>>>>>>>> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>>>>> reason is
>>>>>>>> obvious :)
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> Zhendong
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> -------------------------------------------------------------
>>>>> 
>>>>> Zhen-Dong Zhao (Maxim)
>>>>> 
>>>>> <><<><><><><><><><>><><><><><>>>>>>
>>>>> 
>>>>> Department of Computer Science
>>>>> School of Computing
>>>>> National University of Singapore
>>>>> 
>>>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> -------------------------------------------------------------
>>>> 
>>>> Zhen-Dong Zhao (Maxim)
>>>> 
>>>> <><<><><><><><><><>><><><><><>>>>>>
>>>> 
>>>> Department of Computer Science
>>>> School of Computing
>>>> National University of Singapore
>>>> 
>>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> Ted Dunning, CTO
>> DeepDyve
>> 

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem using Solr/Lucene: http://www.lucidimagination.com/search

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Kay Kay <ka...@gmail.com>.

On 01/23/2010 11:44 AM, Stack wrote:
>> From: Drew Farris<dr...@gmail.com>
>> Date: Sat, Jan 23, 2010 at 8:25 AM
>> Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
>> To: mahout-dev@lucene.apache.org
>>
>>
>> How was the current hbase deployed to the apache repo under the mahout
>> package name? I'd be happy to help get these (and a newer hadoop)
>> rolled in, I'm just unsure of the process.
>>
>>      
> Speaking from hbase, we were not involved -- to the best of my
> knowledge -- adding hbase under mahout package up in the apache hbase
> repo, but all power to who ever did it.
>
> As to hbase showing up in the apache repo., snapshots of hbase trunk
> -- which is built against the hadoop 0.21 branch, which may not be
> what you want -- should be showing up under the hbase package sometime
> in the near future (The issue is
> https://issues.apache.org/jira/browse/HBASE-1933).  In the way are our
> dependency on ZooKeeper and Thrift.  These are not yet available in
> maven repositories (ZooKeeper is planning on publishing the next
> release to maven as Mahadev says below and some fine lads are working
> on getting thrift up into a repo under the guise of
> https://issues.apache.org/jira/browse/THRIFT-363).
>    

THRIFT-363 is more or less complete, with the thrift dev team close to 
approving it.
Now - it is the question of identifying the repo publishing / 
authentication and it should there pretty soon (hopefully !).
If you want to expedite  the process - vote for INFRA-2445 and 
THRIFT-363 and put the nudge there :) .

Meanwhile - if you want to apply to any retroactive versions - feel free 
to apply the latest patch in THRIFT-363 to a repo of your choice and use 
it.


> If someone wants to help out publishing current hbase (0.20.x), let us
> (me) know how we can help.  Regards the zk and thrift dependencies,
> we're toying with putting these up in a temporary repository anyways
> which should do as a stopgap until we all grow into their maven
> deposited versions.
>
> Yours,
> St.Ack
>
>
>    
>> On Fri, Jan 22, 2010 at 5:07 PM, Ted Dunning<te...@gmail.com>  wrote:
>>      
>>> Feedback from ZK and hbase folks:
>>>
>>> Mahadev Konar said:
>>> Unfortunately no.. We are planning to deploy 3.3 as the first version on
>>> maven repo.
>>>
>>> StAck said:
>>> We're working on hbase 0.21 as being the first hbase that shows up in
>>> a maven repo.
>>>
>>> It looks like we are on our own.
>>>
>>> On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris<dr...@gmail.com>
>>> wrote:
>>>
>>>        
>>>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>>>> maven repo at this point, so Mahout would have to roll and deploy
>>>> these. What was the process that was followed to build and deploy the
>>>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>>>> could submit a patch to Mahout for, or better left for the committers?
>>>>
>>>> As Ted pointed out, yes the release of zk is 3.2.2
>>>>
>>>> Drew
>>>>
>>>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong<zh...@gmail.com>
>>>> wrote:
>>>>          
>>>>> Hi Drew,
>>>>>
>>>>> I propose to
>>>>> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is
>>>>> stable
>>>>> and hbased-platform is based on this version,
>>>>>
>>>>> 2) and add zookeeper-3.2.1.jar.
>>>>>
>>>>> Cheers,
>>>>> Zhendong
>>>>>
>>>>> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong<zhaozhendong@gmail.com
>>>>> wrote:
>>>>>
>>>>>            
>>>>>> Hi Drew,
>>>>>>
>>>>>> Including a source code in snapshots that will be great.
>>>>>>
>>>>>> Currently, the HDFS reader does not work in 0.20.2. Without source
>>>>>> code,
>>>>>> it's not convenient for me to debug the code.
>>>>>>
>>>>>> Cheers,
>>>>>> Zhendong
>>>>>>
>>>>>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris<drew.farris@gmail.com
>>>>>>              
>>>>> wrote:
>>>>>            
>>>>>>              
>>>>>>> I wonder if we can get the hadoop people to include source jars with
>>>>>>> their snapshots?
>>>>>>>
>>>>>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen<sr...@gmail.com>  wrote:
>>>>>>>                
>>>>>>>> I need a fix after 0.20.1, that's the primary reason. As a bonus,
>>>>>>>> we
>>>>>>>> don't have to maintain our own version. The downside is relying on
>>>>>>>> a
>>>>>>>> SNAPSHOT, but seems worth it to me.
>>>>>>>>
>>>>>>>> On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong<
>>>>>>>>                  
>>>> zhaozhendong@gmail.com>
>>>>          
>>>>>>> wrote:
>>>>>>>                
>>>>>>>>> Thanks Drew,
>>>>>>>>>
>>>>>>>>> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>>>>>>>>                    
>>>>>>> reason is
>>>>>>>                
>>>>>>>>> obvious :)
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Zhendong
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>>                  
>>>>>>>                
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -------------------------------------------------------------
>>>>>>
>>>>>> Zhen-Dong Zhao (Maxim)
>>>>>>
>>>>>> <><<><><><><><><><>><><><><><>>>>>>
>>>>>>
>>>>>> Department of Computer Science
>>>>>> School of Computing
>>>>>> National University of Singapore
>>>>>>
>>>>>>              
>>>>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>>>>>>>>>>>                            
>>>>>>
>>>>>>
>>>>>>              
>>>>>
>>>>> --
>>>>> -------------------------------------------------------------
>>>>>
>>>>> Zhen-Dong Zhao (Maxim)
>>>>>
>>>>> <><<><><><><><><><>><><><><><>>>>>>
>>>>>
>>>>> Department of Computer Science
>>>>> School of Computing
>>>>> National University of Singapore
>>>>>
>>>>>            
>>>>>>>>>>>> <><><><><><><><<><>><><<<<<<
>>>>>>>>>>>>                          
>>>>>            
>>>>          
>>>
>>>
>>> --
>>> Ted Dunning, CTO
>>> DeepDyve
>>>
>>>        
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>>
>>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Stack <st...@duboce.net>.

> From: Drew Farris <dr...@gmail.com>
> Date: Sat, Jan 23, 2010 at 8:25 AM
> Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
> To: mahout-dev@lucene.apache.org
>
>
> How was the current hbase deployed to the apache repo under the mahout
> package name? I'd be happy to help get these (and a newer hadoop)
> rolled in, I'm just unsure of the process.
>

Speaking from hbase, we were not involved -- to the best of my
knowledge -- adding hbase under mahout package up in the apache hbase
repo, but all power to who ever did it.

As to hbase showing up in the apache repo., snapshots of hbase trunk
-- which is built against the hadoop 0.21 branch, which may not be
what you want -- should be showing up under the hbase package sometime
in the near future (The issue is
https://issues.apache.org/jira/browse/HBASE-1933).  In the way are our
dependency on ZooKeeper and Thrift.  These are not yet available in
maven repositories (ZooKeeper is planning on publishing the next
release to maven as Mahadev says below and some fine lads are working
on getting thrift up into a repo under the guise of
https://issues.apache.org/jira/browse/THRIFT-363).

If someone wants to help out publishing current hbase (0.20.x), let us
(me) know how we can help.  Regards the zk and thrift dependencies,
we're toying with putting these up in a temporary repository anyways
which should do as a stopgap until we all grow into their maven
deposited versions.

Yours,
St.Ack


> On Fri, Jan 22, 2010 at 5:07 PM, Ted Dunning <te...@gmail.com> wrote:
>> Feedback from ZK and hbase folks:
>>
>> Mahadev Konar said:
>> Unfortunately no.. We are planning to deploy 3.3 as the first version on
>> maven repo.
>>
>> StAck said:
>> We're working on hbase 0.21 as being the first hbase that shows up in
>> a maven repo.
>>
>> It looks like we are on our own.
>>
>> On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris <dr...@gmail.com>
>> wrote:
>>
>>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>>> maven repo at this point, so Mahout would have to roll and deploy
>>> these. What was the process that was followed to build and deploy the
>>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>>> could submit a patch to Mahout for, or better left for the committers?
>>>
>>> As Ted pointed out, yes the release of zk is 3.2.2
>>>
>>> Drew
>>>
>>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
>>> wrote:
>>> > Hi Drew,
>>> >
>>> > I propose to
>>> > 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is
>>> > stable
>>> > and hbased-platform is based on this version,
>>> >
>>> > 2) and add zookeeper-3.2.1.jar.
>>> >
>>> > Cheers,
>>> > Zhendong
>>> >
>>> > On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>>> >wrote:
>>> >
>>> >> Hi Drew,
>>> >>
>>> >> Including a source code in snapshots that will be great.
>>> >>
>>> >> Currently, the HDFS reader does not work in 0.20.2. Without source
>>> >> code,
>>> >> it's not convenient for me to debug the code.
>>> >>
>>> >> Cheers,
>>> >> Zhendong
>>> >>
>>> >> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>>> >wrote:
>>> >>
>>> >>> I wonder if we can get the hadoop people to include source jars with
>>> >>> their snapshots?
>>> >>>
>>> >>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>> >>> > I need a fix after 0.20.1, that's the primary reason. As a bonus,
>>> >>> > we
>>> >>> > don't have to maintain our own version. The downside is relying on
>>> >>> > a
>>> >>> > SNAPSHOT, but seems worth it to me.
>>> >>> >
>>> >>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <
>>> zhaozhendong@gmail.com>
>>> >>> wrote:
>>> >>> >> Thanks Drew,
>>> >>> >>
>>> >>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>> >>> reason is
>>> >>> >> obvious :)
>>> >>> >>
>>> >>> >> Cheers,
>>> >>> >> Zhendong
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> -------------------------------------------------------------
>>> >>
>>> >> Zhen-Dong Zhao (Maxim)
>>> >>
>>> >> <><<><><><><><><><>><><><><><>>>>>>
>>> >>
>>> >> Department of Computer Science
>>> >> School of Computing
>>> >> National University of Singapore
>>> >>
>>> >> >>>>>>><><><><><><><><<><>><><<<<<<
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > -------------------------------------------------------------
>>> >
>>> > Zhen-Dong Zhao (Maxim)
>>> >
>>> > <><<><><><><><><><>><><><><><>>>>>>
>>> >
>>> > Department of Computer Science
>>> > School of Computing
>>> > National University of Singapore
>>> >
>>> >>>>>>>><><><><><><><><<><>><><<<<<<
>>> >
>>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
>

Fwd: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Ted Dunning <te...@gmail.com>.

Michael,

Can you follow up on mahout-dev?

---------- Forwarded message ----------
From: Drew Farris <dr...@gmail.com>
Date: Sat, Jan 23, 2010 at 8:25 AM
Subject: Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup
To: mahout-dev@lucene.apache.org


How was the current hbase deployed to the apache repo under the mahout
package name? I'd be happy to help get these (and a newer hadoop)
rolled in, I'm just unsure of the process.

On Fri, Jan 22, 2010 at 5:07 PM, Ted Dunning <te...@gmail.com> wrote:
> Feedback from ZK and hbase folks:
>
> Mahadev Konar said:
> Unfortunately no.. We are planning to deploy 3.3 as the first version on
> maven repo.
>
> StAck said:
> We're working on hbase 0.21 as being the first hbase that shows up in
> a maven repo.
>
> It looks like we are on our own.
>
> On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris <dr...@gmail.com>
wrote:
>
>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>> maven repo at this point, so Mahout would have to roll and deploy
>> these. What was the process that was followed to build and deploy the
>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>> could submit a patch to Mahout for, or better left for the committers?
>>
>> As Ted pointed out, yes the release of zk is 3.2.2
>>
>> Drew
>>
>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
>> wrote:
>> > Hi Drew,
>> >
>> > I propose to
>> > 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is
stable
>> > and hbased-platform is based on this version,
>> >
>> > 2) and add zookeeper-3.2.1.jar.
>> >
>> > Cheers,
>> > Zhendong
>> >
>> > On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>> >wrote:
>> >
>> >> Hi Drew,
>> >>
>> >> Including a source code in snapshots that will be great.
>> >>
>> >> Currently, the HDFS reader does not work in 0.20.2. Without source
code,
>> >> it's not convenient for me to debug the code.
>> >>
>> >> Cheers,
>> >> Zhendong
>> >>
>> >> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>> >wrote:
>> >>
>> >>> I wonder if we can get the hadoop people to include source jars with
>> >>> their snapshots?
>> >>>
>> >>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>> >>> > I need a fix after 0.20.1, that's the primary reason. As a bonus,
we
>> >>> > don't have to maintain our own version. The downside is relying on
a
>> >>> > SNAPSHOT, but seems worth it to me.
>> >>> >
>> >>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <
>> zhaozhendong@gmail.com>
>> >>> wrote:
>> >>> >> Thanks Drew,
>> >>> >>
>> >>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>> >>> reason is
>> >>> >> obvious :)
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Zhendong
>> >>> >>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> -------------------------------------------------------------
>> >>
>> >> Zhen-Dong Zhao (Maxim)
>> >>
>> >> <><<><><><><><><><>><><><><><>>>>>>
>> >>
>> >> Department of Computer Science
>> >> School of Computing
>> >> National University of Singapore
>> >>
>> >> >>>>>>><><><><><><><><<><>><><<<<<<
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > -------------------------------------------------------------
>> >
>> > Zhen-Dong Zhao (Maxim)
>> >
>> > <><<><><><><><><><>><><><><><>>>>>>
>> >
>> > Department of Computer Science
>> > School of Computing
>> > National University of Singapore
>> >
>> >>>>>>>><><><><><><><><<><>><><<<<<<
>> >
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>



-- 
Ted Dunning, CTO
DeepDyve

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Drew Farris <dr...@gmail.com>.

How was the current hbase deployed to the apache repo under the mahout
package name? I'd be happy to help get these (and a newer hadoop)
rolled in, I'm just unsure of the process.

On Fri, Jan 22, 2010 at 5:07 PM, Ted Dunning <te...@gmail.com> wrote:
> Feedback from ZK and hbase folks:
>
> Mahadev Konar said:
> Unfortunately no.. We are planning to deploy 3.3 as the first version on
> maven repo.
>
> StAck said:
> We're working on hbase 0.21 as being the first hbase that shows up in
> a maven repo.
>
> It looks like we are on our own.
>
> On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris <dr...@gmail.com> wrote:
>
>> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
>> maven repo at this point, so Mahout would have to roll and deploy
>> these. What was the process that was followed to build and deploy the
>> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
>> could submit a patch to Mahout for, or better left for the committers?
>>
>> As Ted pointed out, yes the release of zk is 3.2.2
>>
>> Drew
>>
>> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
>> wrote:
>> > Hi Drew,
>> >
>> > I propose to
>> > 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
>> > and hbased-platform is based on this version,
>> >
>> > 2) and add zookeeper-3.2.1.jar.
>> >
>> > Cheers,
>> > Zhendong
>> >
>> > On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
>> >wrote:
>> >
>> >> Hi Drew,
>> >>
>> >> Including a source code in snapshots that will be great.
>> >>
>> >> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>> >> it's not convenient for me to debug the code.
>> >>
>> >> Cheers,
>> >> Zhendong
>> >>
>> >> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
>> >wrote:
>> >>
>> >>> I wonder if we can get the hadoop people to include source jars with
>> >>> their snapshots?
>> >>>
>> >>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>> >>> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>> >>> > don't have to maintain our own version. The downside is relying on a
>> >>> > SNAPSHOT, but seems worth it to me.
>> >>> >
>> >>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <
>> zhaozhendong@gmail.com>
>> >>> wrote:
>> >>> >> Thanks Drew,
>> >>> >>
>> >>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>> >>> reason is
>> >>> >> obvious :)
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Zhendong
>> >>> >>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> -------------------------------------------------------------
>> >>
>> >> Zhen-Dong Zhao (Maxim)
>> >>
>> >> <><<><><><><><><><>><><><><><>>>>>>
>> >>
>> >> Department of Computer Science
>> >> School of Computing
>> >> National University of Singapore
>> >>
>> >> >>>>>>><><><><><><><><<><>><><<<<<<
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > -------------------------------------------------------------
>> >
>> > Zhen-Dong Zhao (Maxim)
>> >
>> > <><<><><><><><><><>><><><><><>>>>>>
>> >
>> > Department of Computer Science
>> > School of Computing
>> > National University of Singapore
>> >
>> >>>>>>>><><><><><><><><<><>><><<<<<<
>> >
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Ted Dunning <te...@gmail.com>.

Feedback from ZK and hbase folks:

Mahadev Konar said:
Unfortunately no.. We are planning to deploy 3.3 as the first version on
maven repo.

StAck said:
We're working on hbase 0.21 as being the first hbase that shows up in
a maven repo.

It looks like we are on our own.

On Fri, Jan 22, 2010 at 11:47 AM, Drew Farris <dr...@gmail.com> wrote:

> Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
> maven repo at this point, so Mahout would have to roll and deploy
> these. What was the process that was followed to build and deploy the
> mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
> could submit a patch to Mahout for, or better left for the committers?
>
> As Ted pointed out, yes the release of zk is 3.2.2
>
> Drew
>
> On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com>
> wrote:
> > Hi Drew,
> >
> > I propose to
> > 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
> > and hbased-platform is based on this version,
> >
> > 2) and add zookeeper-3.2.1.jar.
> >
> > Cheers,
> > Zhendong
> >
> > On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zhaozhendong@gmail.com
> >wrote:
> >
> >> Hi Drew,
> >>
> >> Including a source code in snapshots that will be great.
> >>
> >> Currently, the HDFS reader does not work in 0.20.2. Without source code,
> >> it's not convenient for me to debug the code.
> >>
> >> Cheers,
> >> Zhendong
> >>
> >> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <drew.farris@gmail.com
> >wrote:
> >>
> >>> I wonder if we can get the hadoop people to include source jars with
> >>> their snapshots?
> >>>
> >>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
> >>> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
> >>> > don't have to maintain our own version. The downside is relying on a
> >>> > SNAPSHOT, but seems worth it to me.
> >>> >
> >>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <
> zhaozhendong@gmail.com>
> >>> wrote:
> >>> >> Thanks Drew,
> >>> >>
> >>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
> >>> reason is
> >>> >> obvious :)
> >>> >>
> >>> >> Cheers,
> >>> >> Zhendong
> >>> >>
> >>> >>
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> -------------------------------------------------------------
> >>
> >> Zhen-Dong Zhao (Maxim)
> >>
> >> <><<><><><><><><><>><><><><><>>>>>>
> >>
> >> Department of Computer Science
> >> School of Computing
> >> National University of Singapore
> >>
> >> >>>>>>><><><><><><><><<><>><><<<<<<
> >>
> >>
> >>
> >
> >
> > --
> > -------------------------------------------------------------
> >
> > Zhen-Dong Zhao (Maxim)
> >
> > <><<><><><><><><><>><><><><><>>>>>>
> >
> > Department of Computer Science
> > School of Computing
> > National University of Singapore
> >
> >>>>>>>><><><><><><><><<><>><><<<<<<
> >
>



-- 
Ted Dunning, CTO
DeepDyve

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Drew Farris <dr...@gmail.com>.

Neither hbase 0.20.2 nor zookeeper (any version) appear to be in a
maven repo at this point, so Mahout would have to roll and deploy
these. What was the process that was followed to build and deploy the
mahout-packaged hadoop 0.20.1 and hbase artifacts? Is this something I
could submit a patch to Mahout for, or better left for the committers?

As Ted pointed out, yes the release of zk is 3.2.2

Drew

On Thu, Jan 21, 2010 at 5:12 AM, zhao zhendong <zh...@gmail.com> wrote:
> Hi Drew,
>
> I propose to
> 1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
> and hbased-platform is based on this version,
>
> 2) and add zookeeper-3.2.1.jar.
>
> Cheers,
> Zhendong
>
> On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zh...@gmail.com>wrote:
>
>> Hi Drew,
>>
>> Including a source code in snapshots that will be great.
>>
>> Currently, the HDFS reader does not work in 0.20.2. Without source code,
>> it's not convenient for me to debug the code.
>>
>> Cheers,
>> Zhendong
>>
>> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <dr...@gmail.com>wrote:
>>
>>> I wonder if we can get the hadoop people to include source jars with
>>> their snapshots?
>>>
>>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>>> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>>> > don't have to maintain our own version. The downside is relying on a
>>> > SNAPSHOT, but seems worth it to me.
>>> >
>>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
>>> wrote:
>>> >> Thanks Drew,
>>> >>
>>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>>> reason is
>>> >> obvious :)
>>> >>
>>> >> Cheers,
>>> >> Zhendong
>>> >>
>>> >>
>>> >
>>>
>>
>>
>>
>> --
>> -------------------------------------------------------------
>>
>> Zhen-Dong Zhao (Maxim)
>>
>> <><<><><><><><><><>><><><><><>>>>>>
>>
>> Department of Computer Science
>> School of Computing
>> National University of Singapore
>>
>> >>>>>>><><><><><><><><<><>><><<<<<<
>>
>>
>>
>
>
> --
> -------------------------------------------------------------
>
> Zhen-Dong Zhao (Maxim)
>
> <><<><><><><><><><>><><><><><>>>>>>
>
> Department of Computer Science
> School of Computing
> National University of Singapore
>
>>>>>>>><><><><><><><><<><>><><<<<<<
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by zhao zhendong <zh...@gmail.com>.

Hi Drew,

I propose to
1) update hbase-0.20.0.jar to hbase-0.20.2.jar due to the later is stable
and hbased-platform is based on this version,

2) and add zookeeper-3.2.1.jar.

Cheers,
Zhendong

On Tue, Jan 19, 2010 at 12:36 PM, zhao zhendong <zh...@gmail.com>wrote:

> Hi Drew,
>
> Including a source code in snapshots that will be great.
>
> Currently, the HDFS reader does not work in 0.20.2. Without source code,
> it's not convenient for me to debug the code.
>
> Cheers,
> Zhendong
>
> On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <dr...@gmail.com>wrote:
>
>> I wonder if we can get the hadoop people to include source jars with
>> their snapshots?
>>
>> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
>> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
>> > don't have to maintain our own version. The downside is relying on a
>> > SNAPSHOT, but seems worth it to me.
>> >
>> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
>> wrote:
>> >> Thanks Drew,
>> >>
>> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
>> reason is
>> >> obvious :)
>> >>
>> >> Cheers,
>> >> Zhendong
>> >>
>> >>
>> >
>>
>
>
>
> --
> -------------------------------------------------------------
>
> Zhen-Dong Zhao (Maxim)
>
> <><<><><><><><><><>><><><><><>>>>>>
>
> Department of Computer Science
> School of Computing
> National University of Singapore
>
> >>>>>>><><><><><><><><<><>><><<<<<<
>
>
>


-- 
-------------------------------------------------------------

Zhen-Dong Zhao (Maxim)

<><<><><><><><><><>><><><><><>>>>>>

Department of Computer Science
School of Computing
National University of Singapore

>>>>>>><><><><><><><><<><>><><<<<<<

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by zhao zhendong <zh...@gmail.com>.

Hi Drew,

Including a source code in snapshots that will be great.

Currently, the HDFS reader does not work in 0.20.2. Without source code,
it's not convenient for me to debug the code.

Cheers,
Zhendong

On Sat, Jan 9, 2010 at 12:25 AM, Drew Farris <dr...@gmail.com> wrote:

> I wonder if we can get the hadoop people to include source jars with
> their snapshots?
>
> On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
> > I need a fix after 0.20.1, that's the primary reason. As a bonus, we
> > don't have to maintain our own version. The downside is relying on a
> > SNAPSHOT, but seems worth it to me.
> >
> > On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com>
> wrote:
> >> Thanks Drew,
> >>
> >> +1 for me to maintain a stable hadoop release, such as 0.20.1. The
> reason is
> >> obvious :)
> >>
> >> Cheers,
> >> Zhendong
> >>
> >>
> >
>



-- 
-------------------------------------------------------------

Zhen-Dong Zhao (Maxim)

<><<><><><><><><><>><><><><><>>>>>>

Department of Computer Science
School of Computing
National University of Singapore

>>>>>>><><><><><><><><<><>><><<<<<<

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Drew Farris <dr...@gmail.com>.

I wonder if we can get the hadoop people to include source jars with
their snapshots?

On Fri, Jan 8, 2010 at 11:23 AM, Sean Owen <sr...@gmail.com> wrote:
> I need a fix after 0.20.1, that's the primary reason. As a bonus, we
> don't have to maintain our own version. The downside is relying on a
> SNAPSHOT, but seems worth it to me.
>
> On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com> wrote:
>> Thanks Drew,
>>
>> +1 for me to maintain a stable hadoop release, such as 0.20.1. The reason is
>> obvious :)
>>
>> Cheers,
>> Zhendong
>>
>>
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Sean Owen <sr...@gmail.com>.

I need a fix after 0.20.1, that's the primary reason. As a bonus, we
don't have to maintain our own version. The downside is relying on a
SNAPSHOT, but seems worth it to me.

On Fri, Jan 8, 2010 at 4:02 PM, zhao zhendong <zh...@gmail.com> wrote:
> Thanks Drew,
>
> +1 for me to maintain a stable hadoop release, such as 0.20.1. The reason is
> obvious :)
>
> Cheers,
> Zhendong
>
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by zhao zhendong <zh...@gmail.com>.

Thanks Drew,

+1 for me to maintain a stable hadoop release, such as 0.20.1. The reason is
obvious :)

Cheers,
Zhendong


On Fri, Jan 8, 2010 at 10:23 PM, Drew Farris <dr...@gmail.com> wrote:

> First, apologies for propagating a problem here.
>
> Since 0.20.2 is a snapshot, there's no release of hadoop that
> corresponds directly to it. In maven terms, a snapshot could be
> anything after 0.20.1 but prior to a formal release of 0.20.2. In the
> repo, they are timestamped.
>
> We're pulling this jar from the apache snapsot repo here:
>
> https://repository.apache.org/content/groups/snapshots/org/apache/hadoop/hadoop-core/0.20.2-SNAPSHOT/
>
> Unfortunately, hadoop doesn't publish a source jar alongisde their
> snapshot, which makes it a bit difficut to pin down what was used for
> the build of jar. From the maven repo, you can see the jar was
> generated 2009.11.13, 1:09:47 GMT, so as Ted suggests one option would
> be to check out the hadoop-core sources from that date and use those
> as a basis for your work on MAHOUT-232
>
> I'm not 100% comfortable with this. When I picked up the work for
> MAHOUT-238, I noticed that hadoop-0.20.2 SNAPSHOT was in core yet
> hadoop-0.20.1, the mahout hand-rolled version, was used in math and
> thus propagated as a transient dependency across all of the projects.
>
> I believe we moved to a SNAPSHOT so that we could use one of the
> offical hadoop dependencies instead of our hand-rolled version of
> 0.20.1. IIRC, there are some bugfixes that we required to, but I'm not
> certain about that?
>
> Does anyone feel that we should move back 0.20.1 and use a stable
> hadoop release instead of using the snapshot, so that the version of
> the code we depend on is clearer?
>
> Drew
>
> On Fri, Jan 8, 2010 at 2:08 AM, Ted Dunning <te...@gmail.com> wrote:
> > SVN should help: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/
> >
> > If you look at the release notes, you should be able to discern what made
> up
> > 20.2 if it is a real release (looking at common made is look like it
> isn't).
> >
> > On Thu, Jan 7, 2010 at 9:46 PM, zhao zhendong <zhaozhendong@gmail.com
> >wrote:
> >
> >> Where can I find the source code of hadoop-0.20.2?
> >
> >
> >
> >
> > --
> > Ted Dunning, CTO
> > DeepDyve
> >
>



-- 
-------------------------------------------------------------

Zhen-Dong Zhao (Maxim)

<><<><><><><><><><>><><><><><>>>>>>

Department of Computer Science
School of Computing
National University of Singapore

><><><><><><><><><><><><><><><><<<<
Homepage:http://zhaozhendong.googlepages.com
Mail: zhaozhendong@gmail.com
>>>>>>><><><><><><><><<><>><><<<<<<

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Drew Farris <dr...@gmail.com>.

First, apologies for propagating a problem here.

Since 0.20.2 is a snapshot, there's no release of hadoop that
corresponds directly to it. In maven terms, a snapshot could be
anything after 0.20.1 but prior to a formal release of 0.20.2. In the
repo, they are timestamped.

We're pulling this jar from the apache snapsot repo here:
https://repository.apache.org/content/groups/snapshots/org/apache/hadoop/hadoop-core/0.20.2-SNAPSHOT/

Unfortunately, hadoop doesn't publish a source jar alongisde their
snapshot, which makes it a bit difficut to pin down what was used for
the build of jar. From the maven repo, you can see the jar was
generated 2009.11.13, 1:09:47 GMT, so as Ted suggests one option would
be to check out the hadoop-core sources from that date and use those
as a basis for your work on MAHOUT-232

I'm not 100% comfortable with this. When I picked up the work for
MAHOUT-238, I noticed that hadoop-0.20.2 SNAPSHOT was in core yet
hadoop-0.20.1, the mahout hand-rolled version, was used in math and
thus propagated as a transient dependency across all of the projects.

I believe we moved to a SNAPSHOT so that we could use one of the
offical hadoop dependencies instead of our hand-rolled version of
0.20.1. IIRC, there are some bugfixes that we required to, but I'm not
certain about that?

Does anyone feel that we should move back 0.20.1 and use a stable
hadoop release instead of using the snapshot, so that the version of
the code we depend on is clearer?

Drew

On Fri, Jan 8, 2010 at 2:08 AM, Ted Dunning <te...@gmail.com> wrote:
> SVN should help: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/
>
> If you look at the release notes, you should be able to discern what made up
> 20.2 if it is a real release (looking at common made is look like it isn't).
>
> On Thu, Jan 7, 2010 at 9:46 PM, zhao zhendong <zh...@gmail.com>wrote:
>
>> Where can I find the source code of hadoop-0.20.2?
>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by Ted Dunning <te...@gmail.com>.

SVN should help: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/

If you look at the release notes, you should be able to discern what made up
20.2 if it is a real release (looking at common made is look like it isn't).

On Thu, Jan 7, 2010 at 9:46 PM, zhao zhendong <zh...@gmail.com>wrote:

> Where can I find the source code of hadoop-0.20.2?




-- 
Ted Dunning, CTO
DeepDyve

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

Posted by zhao zhendong <zh...@gmail.com>.

Hi Drew,

Where can I find the source code of hadoop-0.20.2?

Parts of MAHOUT-232 code depend on hadoop-0.20.1 reducer.context. Hadoop
team seems to modify this API in hadoop-0.20.2, and I CANNOT find the source
code of Hadoop-0.20.2, thus it's impossible for me to update my code.

Thanks.

On Thu, Jan 7, 2010 at 9:54 PM, Drew Farris (JIRA) <ji...@apache.org> wrote:

>
>    [
> https://issues.apache.org/jira/browse/MAHOUT-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797627#action_12797627]
>
> Drew Farris commented on MAHOUT-238:
> ------------------------------------
>
> bq. For some reason I can't apply the patch but again suspect it's a local
> problem. I'm about to just blow this all away and start over.
>
> Does is apply cleanly to a fresh checkout when patching from the
> command-line? If you give it another try and it still fails, I'll look into
> it further.
>
> bq. we shouldn't directly depend on commons-logging right? we log via SLF4J
> only.
>
> Right, we log via the slf4j api, but at runtime, the slf4j uses
> commons-logging to do its work. slf4j-api -> slf4j-jcl -> commons-logging.
>
> I'm not exactly sure why that particular mechanism was chosen for mahout,
> but the most common case for that route is when one of our dependencies
> depend on commons-logging and it needs to be in the classpath anyway.
>
> That aside, it would probably make sense to change the scope of
> commons-logging to runtime. It is certainly not needed for compilation.
> After thinking about this a bit, I suspect I probably need to change
> slf4j-jcl to a runtime dependency as well (as opposed to a test dependency).
> This way, they are there as dependencies if they are needed, but if someone
> using mahout wants to use a different framework they can be excluded.
>
>
>
>
>
>
>
>
> > Further Dependency Cleanup
> > --------------------------
> >
> >                 Key: MAHOUT-238
> >                 URL: https://issues.apache.org/jira/browse/MAHOUT-238
> >             Project: Mahout
> >          Issue Type: Sub-task
> >    Affects Versions: 0.2
> >            Reporter: Drew Farris
> >            Priority: Minor
> >             Fix For: 0.3
> >
> >         Attachments: MAHOUT-238.patch
> >
> >
> > Further dependency cleanup is required, mainly to set the right hadoop
> dependency for mahout-math and fix exclusions for the hadoop dependency in
> the parent pom. Other minor cleanups too.
> > The patch includes the following changes:
> > maven (parent pom)
> > * added inceptionYear (2008)
> > * removed some exclusions for hadoop dependency: avro, commons-codec,
> commons-httpclient in the dependendy management section.
> > * removed javax.mail dependency
> > mahout-math
> > * switched from o.a.m.hadoop:hadoop-core dependency to new
> o.a.hadoop:hadoop-core dependency used in core, version specified in
> dependencyManagement section of parent pom.
> > * removed unnecessary compile scope from gson dependency
> > mahout-core
> > * removed: kfs, jets3t, xmlenc, unused, originally added to support old
> o.a.mahout.hadoop:hadoop-core:0.20.1 dependency
> > * removed: commons-httpclient, now added transitively from new
> o.a.hadoop:hadoop-core:0.20.2-SNAPSHOT dependency
> > * set slf4j-jcl to test scope.
> > * removed: watchmaker-swing, added later in mahout-examples where it is
> actually used.
> > * fixed uncommons-maths groupId
> > * removed unused lucene-analyzers dependency.
> > * added easymock dependencies explicitly
> > mahout-utils
> > * removed unused easymock dependencies
> > mahout-examples
> > * added watchmaker-framework and watchmaker-swing
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>


-- 
-------------------------------------------------------------

Zhen-Dong Zhao (Maxim)

<><<><><><><><><><>><><><><><>>>>>>

Department of Computer Science
School of Computing
National University of Singapore

><><><><><><><><><><><><><><><><<<<
Homepage:http://zhaozhendong.googlepages.com
Mail: zhaozhendong@gmail.com
>>>>>>><><><><><><><><<><>><><<<<<<