You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Jukka Zitting <ju...@gmail.com> on 2009/11/11 21:42:45 UTC
Dependencies outside Maven central (Was: Oh joy)
Hi,
On Wed, Nov 11, 2009 at 8:50 PM, Grant Ingersoll <gs...@apache.org> wrote:
> On Nov 11, 2009, at 1:45 PM, Jukka Zitting wrote:
>> The standard and by far the easiest way to get this dependency
>> included in the Mahout build is to ask the Hadoop project do deploy it
>> under the org.apache.hadoop groupId.
>
> They are working on it, but don't have it for our versions.
I pinged the Hadoop dev list about this, see
http://markmail.org/message/643sxbyjcrjxaglt. Hopefully that'll speed
things up, though for Mahout 0.2 the dependency may still need to be
deployed to org.apache.mahout.hadoop.
> We also have other non-Hadoop deps that aren't published, so they
> need to be deployed and signed as well (Commons CLI2).
I asked Commons for a CLI 2.0 release, see
http://markmail.org/message/h357fauzs4arxfsg, though for now I think
Mahout can simply use the already deployed
org.apache.mahout.commons:commons-cli:2.0-mahout version.
I also tracked down the other dependencies that we now try to sign-and-deploy:
* kfs-0.3.jar -> net.sf.kosmosfs:kfs:0.3
* jets3t-0.7.1.jar -> net.java.dev.jets3t:jets3t:0.7.1
* watchmaker-framework-0.6.2.jar -> not available
* watchmaker-swing-0.6.2.jar -> not available
* uncommons-maths-1.2.jar -> not availabe
It should be possible to deploy the missing artifacts like this:
mvn gpg:sign-and-deploy-file \
-Durl=https://repository.apache.org/service/local/staging/deploy/maven2
\
-DrepositoryId=apache.releases.https \
-DgroupId=org.apache.mahout.xxx -DartifactId=xxx \
-Dversion=xxx -Dpackaging=jar -Dfile=core/lib/xxx.jar
BR,
Jukka Zitting
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Isabel Drost <is...@apache.org>.
On Thu Grant Ingersoll <gs...@apache.org> wrote:
> That's weird, that is the default Maven repository, I wouldn't think
> you would need to add it.
Hmm. Right. That is weird.
Isabel
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
That's weird, that is the default Maven repository, I wouldn't think
you would need to add it.
On Nov 12, 2009, at 5:30 AM, Isabel Drost wrote:
> On Wed, 11 Nov 2009 18:23:50 -0500
> Grant Ingersoll <gs...@apache.org> wrote:
>
>> https://issues.apache.org/jira/browse/MAHOUT-198 tracks this. I am
>> committing now. Please check out, delete your ~/.m2/repository/org
>> directory and try mvn clean install!
>
> Trashed my local maven repo, checked out an built again - it did not
> find the lucene-2.9.1 release. Adding the following repository to the
> pom fixed that problem for me:
>
> Index: maven/pom.xml
> ===================================================================
> --- maven/pom.xml (revision 835320)
> +++ maven/pom.xml (working copy)
> @@ -76,6 +76,12 @@
> <layout>default</layout>
> </repository>
> <repository>
> + <id>maven2-repository.maven.org</id>
> + <name>Maven.org Repository for Maven</name>
> + <url>http://repo1.maven.org/maven2</url>
> + <layout>default</layout>
> + </repository>
> + <repository>
> <id>Apache snapshots</id>
> <url>http://people.apache.org/maven-snapshot-repository</url>
> <snapshots>
>
> After that the build runs smoothly for me. Big Thanks to you Grant,
> for
> resolving all those issues.
>
> Isabel
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Isabel Drost <is...@apache.org>.
On Wed, 11 Nov 2009 18:23:50 -0500
Grant Ingersoll <gs...@apache.org> wrote:
> https://issues.apache.org/jira/browse/MAHOUT-198 tracks this. I am
> committing now. Please check out, delete your ~/.m2/repository/org
> directory and try mvn clean install!
Trashed my local maven repo, checked out an built again - it did not
find the lucene-2.9.1 release. Adding the following repository to the
pom fixed that problem for me:
Index: maven/pom.xml
===================================================================
--- maven/pom.xml (revision 835320)
+++ maven/pom.xml (working copy)
@@ -76,6 +76,12 @@
<layout>default</layout>
</repository>
<repository>
+ <id>maven2-repository.maven.org</id>
+ <name>Maven.org Repository for Maven</name>
+ <url>http://repo1.maven.org/maven2</url>
+ <layout>default</layout>
+ </repository>
+ <repository>
<id>Apache snapshots</id>
<url>http://people.apache.org/maven-snapshot-repository</url>
<snapshots>
After that the build runs smoothly for me. Big Thanks to you Grant, for
resolving all those issues.
Isabel
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
https://issues.apache.org/jira/browse/MAHOUT-198 tracks this. I am
committing now. Please check out, delete your ~/.m2/repository/org
directory and try mvn clean install!
-Grant
On Nov 11, 2009, at 6:07 PM, Grant Ingersoll wrote:
> For the record, here's what I did:
> 1016 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.kosmosfs -DartifactId=kfs -Dversion=0.3 -
> Dpackaging=jar -Dfile=kfs-0.3.jar
> 1017 hg sign-and
> 1018 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.kosmosfs -DartifactId=kfs -Dversion=0.3 -
> Dpackaging=jar -Dfile=kfs-0.3.jar
> 1019 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.hbase -DartifactId=hbase -
> Dversion=0.20.0 -Dpackaging=jar -Dfile=hbase-0.20.0.jar
> 1020 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.jets3t -DartifactId=jets3t -
> Dversion=0.7.1 -Dpackaging=jar -Dfile=jets3t-0.7.1.jar
> 1021 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.hadoop -DartifactId=hadoop-core -
> Dversion=0.20.1 -Dpackaging=jar -Dfile=hadoop-core-0.20.1.jar
> 1022 mvn gpg:sign-and-deploy-file -Durl=https://
> repository.apache.org/service/local/staging/deploy/maven2 -
> DrepositoryId=apache.releases.https -
> DgroupId=org.apache.mahout.kosmofs -DartifactId=kfs -Dversion=0.3 -
> Dpackaging=jar -Dfile=kfs-0.3.jar
>
> I then went into Nexus and closed and then promoted the staging
> repository, such that these are now available. I will commit the
> POM changes shortly.
>
> On Nov 11, 2009, at 5:46 PM, Jukka Zitting wrote:
>
>> Hi,
>>
>> On Thu, Nov 12, 2009 at 12:37 AM, Grant Ingersoll <gsingers@apache.org
>> > wrote:
>>> I'm going forward with this, since I see no difference between
>>> doing it up
>>> front and doing it during a release. This clearly falls under the
>>> category
>>> of "why didn't we do this sooner".
>>
>> Exactly. Maven assumes that all non-SNAPSHOT dependencies are already
>> available in the repository, and Just Works (TM) when that's the
>> case.
>> No need for custom install/deploy steps.
>>
>> BTW, this is also Maven users are so eager to see things like Lucene
>> jars on Maven central, and why having the Hadoop project deploy their
>> jars is better than each downstream project deploying their own
>> copies.
>>
>> BR,
>>
>> Jukka Zitting
>
>
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
For the record, here's what I did:
1016 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -
DgroupId=org.apache.mahout.kosmosfs -DartifactId=kfs -Dversion=0.3 -
Dpackaging=jar -Dfile=kfs-0.3.jar
1017 hg sign-and
1018 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -
DgroupId=org.apache.mahout.kosmosfs -DartifactId=kfs -Dversion=0.3 -
Dpackaging=jar -Dfile=kfs-0.3.jar
1019 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -DgroupId=org.apache.mahout.hbase -
DartifactId=hbase -Dversion=0.20.0 -Dpackaging=jar -
Dfile=hbase-0.20.0.jar
1020 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -DgroupId=org.apache.mahout.jets3t
-DartifactId=jets3t -Dversion=0.7.1 -Dpackaging=jar -
Dfile=jets3t-0.7.1.jar
1021 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -DgroupId=org.apache.mahout.hadoop
-DartifactId=hadoop-core -Dversion=0.20.1 -Dpackaging=jar -
Dfile=hadoop-core-0.20.1.jar
1022 mvn gpg:sign-and-deploy-file -Durl=https://
repository.apache.org/service/local/staging/deploy/maven2 -
DrepositoryId=apache.releases.https -
DgroupId=org.apache.mahout.kosmofs -DartifactId=kfs -Dversion=0.3 -
Dpackaging=jar -Dfile=kfs-0.3.jar
I then went into Nexus and closed and then promoted the staging
repository, such that these are now available. I will commit the POM
changes shortly.
On Nov 11, 2009, at 5:46 PM, Jukka Zitting wrote:
> Hi,
>
> On Thu, Nov 12, 2009 at 12:37 AM, Grant Ingersoll
> <gs...@apache.org> wrote:
>> I'm going forward with this, since I see no difference between
>> doing it up
>> front and doing it during a release. This clearly falls under the
>> category
>> of "why didn't we do this sooner".
>
> Exactly. Maven assumes that all non-SNAPSHOT dependencies are already
> available in the repository, and Just Works (TM) when that's the case.
> No need for custom install/deploy steps.
>
> BTW, this is also Maven users are so eager to see things like Lucene
> jars on Maven central, and why having the Hadoop project deploy their
> jars is better than each downstream project deploying their own
> copies.
>
> BR,
>
> Jukka Zitting
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Nov 12, 2009 at 12:37 AM, Grant Ingersoll <gs...@apache.org> wrote:
> I'm going forward with this, since I see no difference between doing it up
> front and doing it during a release. This clearly falls under the category
> of "why didn't we do this sooner".
Exactly. Maven assumes that all non-SNAPSHOT dependencies are already
available in the repository, and Just Works (TM) when that's the case.
No need for custom install/deploy steps.
BTW, this is also Maven users are so eager to see things like Lucene
jars on Maven central, and why having the Hadoop project deploy their
jars is better than each downstream project deploying their own
copies.
BR,
Jukka Zitting
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 11, 2009, at 5:29 PM, Grant Ingersoll wrote:
>
> On Nov 11, 2009, at 3:42 PM, Jukka Zitting wrote:
>> It should be possible to deploy the missing artifacts like this:
>>
>> mvn gpg:sign-and-deploy-file \
>> -Durl=https://repository.apache.org/service/local/staging/
>> deploy/maven2
>> \
>> -DrepositoryId=apache.releases.https \
>> -DgroupId=org.apache.mahout.xxx -DartifactId=xxx \
>> -Dversion=xxx -Dpackaging=jar -Dfile=core/lib/xxx.jar
>
> Some further thoughts on this:
>
> Perhaps we should do this upfront for any/all deps we use that
> aren't in Maven that we need, regardless of a release. Then we
> don't have to have users do a "mvn install" first, since all
> artifacts will be resolvable.
>
> As Jukka said, then we just need to deploy them once whenever we
> upgrade them. The release process should also get a lot easier.
>
> Any reason not to?
>
I'm going forward with this, since I see no difference between doing
it up front and doing it during a release. This clearly falls under
the category of "why didn't we do this sooner". Artifacts being
signed and uploaded now.
-Grant
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 11, 2009, at 3:42 PM, Jukka Zitting wrote:
> It should be possible to deploy the missing artifacts like this:
>
> mvn gpg:sign-and-deploy-file \
> -Durl=https://repository.apache.org/service/local/staging/
> deploy/maven2
> \
> -DrepositoryId=apache.releases.https \
> -DgroupId=org.apache.mahout.xxx -DartifactId=xxx \
> -Dversion=xxx -Dpackaging=jar -Dfile=core/lib/xxx.jar
Some further thoughts on this:
Perhaps we should do this upfront for any/all deps we use that aren't
in Maven that we need, regardless of a release. Then we don't have to
have users do a "mvn install" first, since all artifacts will be
resolvable.
As Jukka said, then we just need to deploy them once whenever we
upgrade them. The release process should also get a lot easier.
Any reason not to?
-Grant
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 11, 2009, at 4:03 PM, Grant Ingersoll wrote:
>
> On Nov 11, 2009, at 3:42 PM, Jukka Zitting wrote:
>
>>
>> * kfs-0.3.jar -> net.sf.kosmosfs:kfs:0.3
>> * jets3t-0.7.1.jar -> net.java.dev.jets3t:jets3t:0.7.1
>
> Right
>
>> * watchmaker-framework-0.6.2.jar -> not available
>> * watchmaker-swing-0.6.2.jar -> not available
>> * uncommons-maths-1.2.jar -> not availabe
>
> These are now available.
In the Java.net repository. I'm working on moving us over to the new
deps right now. Might as well just make it easier. Then, I should
only have to upload the Hadoop deps.
Re: Dependencies outside Maven central (Was: Oh joy)
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 11, 2009, at 3:42 PM, Jukka Zitting wrote:
>
> * kfs-0.3.jar -> net.sf.kosmosfs:kfs:0.3
> * jets3t-0.7.1.jar -> net.java.dev.jets3t:jets3t:0.7.1
Right
> * watchmaker-framework-0.6.2.jar -> not available
> * watchmaker-swing-0.6.2.jar -> not available
> * uncommons-maths-1.2.jar -> not availabe
These are now available.