You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jason Rosenberg <jb...@squareup.com> on 2013/05/07 01:31:50 UTC
maven integration in 0.8
Is there a maven repo we can point to, to just depend on the kafka 0.8
core, and have all the dependencies get pulled in as needed?
For some reason, I had thought this would be available as part of the 0.8
release....
Or do I need to manually create a pom.xml for the core, and host it on my
nexxus instance (as in 0.7.2?).
Thanks,
Jason
Re: maven integration in 0.8
Posted by Jason Rosenberg <jb...@squareup.com>.
Perhaps, but you don't want to version your kafka artifacts as just 2.8.0.
The way you handle something like that with maven, is to include the
dependency on scala 2.8.0 in the the kafka pom's dependency list.
e.g.:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0</version>
</dependency>
But the version of kafka is 0.8.0, not 2.8.0. Otherwise, how is one to
distinguish it from a kafka 0.7.2 jar?
What I've done now, is to actually include the 2.8.0 in the artifactId, but
not the version id, so:
<groupId>org.apache</groupId>
<artifactId>kafka_2.8.0</artifactId>
<packaging>jar</packaging>
<description>kafka</description>
<version>0.8.0-SHA</version>
<name>kafka</name>
So, in the maven repo it looks like:
/org/apache/kafka_2.8.0/kafka_2.8.0-0.8.0-SHA.jar
Make sense?
I'm contemplating removing the '_2.8.0' bit though, since I don't think it
makes sense there, more I think about it.
I replace the 'SHA' part with the actual git sha of the current build (once
we have a release, I can remove the SHA part).
Jason
On Tue, May 7, 2013 at 3:00 PM, Alex Zuzin <ca...@gmail.com> wrote:
> It's actually a rather helpful convention SBT has instituted to deal with
> bytecode incompatibilities between the different versions of Scala. I
> wouldn't advise removing it at all - using a 2.8.0-built project in
> production with Scala 2.9 or higher is courting disaster.
>
> --
> Alex Zuzin
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
>
> On Tuesday, May 7, 2013 at 14:26, Jason Rosenberg wrote:
>
> > Except that it still has the annoying feature of naming the kafka version
> > after the scala version, which doesn't make sense, e.g.:
> >
> > ~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT/poms ls -l
> > total 24
> > -rw-r--r-- 1 jbr jbr 3786 May 7 13:43 kafka_2.8.0.pom
> > -rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.pom.md5
> > -rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.pom.sha1
> >
> > and
> >
> > ~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT ls -l jars
> > total 4640
> > -rw-r--r-- 1 jbr jbr 2367457 May 6 15:54 kafka_2.8.0.jar
> > -rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.jar.md5
> > -rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.jar.sha1
> >
> > I think it was mentioned previously (0.7.x had the same issue) there's an
> > easy way to fix this (but I don't remember at the moment). Perhaps we
> > could get this fixed before releasing?
> >
> > I'd think the maven artifacts should be named 'kafka_0.8.0.pom' or
> > 'kakfa_0.8.0-SNAPSHOT.pom', etc...
> >
> > Jason
> >
> >
> >
> >
> > On Tue, May 7, 2013 at 1:48 PM, Jason Rosenberg <jb...@squareup.com>
> wrote:
> >
> > > Alex,
> > >
> > > Thanks for the tip, this is exactly what I need.
> > >
> > > Jason
> > >
> > >
> > > On Tue, May 7, 2013 at 11:58 AM, Gray, Alex <Al...@inin.com>
> wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > I don't know if this will help you, but I'm pulling the source code
> of
> > > > kafka down to a local machine and calling:
> > > > ./sbt update
> > > > /sbt -Dsbt.log.noformat=true publish-local
> > > > This will publish the generated pom's and jars to your local ~/.ivy2
> > > > directory.
> > > >
> > > > Once you have that, you can publish those same artifacts to your
> internal
> > > > repo and modify your project's pom to download the artifacts from
> your
> > > > internal repo.
> > > >
> > > > That way, everyone in my group is using the same kafka artifacts
> that I
> > > > am building.
> > > >
> > > >
> > > > Alex Gray
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com]
> > > > Sent: Monday, May 06, 2013 10:27 PM
> > > > To: users@kafka.apache.org
> > > > Subject: Re: maven integration in 0.8
> > > >
> > > > There are a lot of release-oriented tickets still about- here's what
> I'm
> > > > following pertaining to packaging & getting
> > > > shipped-
> > > >
> > > > Maven-
> > > >
> > > >
> https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > > Scala 2.10-
> > > >
> > > >
> https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > > Deps upgrade-
> > > >
> > > >
> https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > > CodaMetrics versioning-
> > > >
> > > >
> https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > >
> > > > On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
> > > > > Is there a maven repo we can point to, to just depend on the kafka
> 0.8
> > > > > core, and have all the dependencies get pulled in as needed?
> > > > >
> > > > > For some reason, I had thought this would be available as part of
> the
> > > > > 0.8 release....
> > > > >
> > > > > Or do I need to manually create a pom.xml for the core, and host
> it on
> > > > > my nexxus instance (as in 0.7.2?).
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jason
>
>
Re: maven integration in 0.8
Posted by Alex Zuzin <ca...@gmail.com>.
It's actually a rather helpful convention SBT has instituted to deal with bytecode incompatibilities between the different versions of Scala. I wouldn't advise removing it at all - using a 2.8.0-built project in production with Scala 2.9 or higher is courting disaster.
--
Alex Zuzin
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Tuesday, May 7, 2013 at 14:26, Jason Rosenberg wrote:
> Except that it still has the annoying feature of naming the kafka version
> after the scala version, which doesn't make sense, e.g.:
>
> ~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT/poms ls -l
> total 24
> -rw-r--r-- 1 jbr jbr 3786 May 7 13:43 kafka_2.8.0.pom
> -rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.pom.md5
> -rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.pom.sha1
>
> and
>
> ~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT ls -l jars
> total 4640
> -rw-r--r-- 1 jbr jbr 2367457 May 6 15:54 kafka_2.8.0.jar
> -rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.jar.md5
> -rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.jar.sha1
>
> I think it was mentioned previously (0.7.x had the same issue) there's an
> easy way to fix this (but I don't remember at the moment). Perhaps we
> could get this fixed before releasing?
>
> I'd think the maven artifacts should be named 'kafka_0.8.0.pom' or
> 'kakfa_0.8.0-SNAPSHOT.pom', etc...
>
> Jason
>
>
>
>
> On Tue, May 7, 2013 at 1:48 PM, Jason Rosenberg <jb...@squareup.com> wrote:
>
> > Alex,
> >
> > Thanks for the tip, this is exactly what I need.
> >
> > Jason
> >
> >
> > On Tue, May 7, 2013 at 11:58 AM, Gray, Alex <Al...@inin.com> wrote:
> >
> > > Hi Jason,
> > >
> > > I don't know if this will help you, but I'm pulling the source code of
> > > kafka down to a local machine and calling:
> > > ./sbt update
> > > /sbt -Dsbt.log.noformat=true publish-local
> > > This will publish the generated pom's and jars to your local ~/.ivy2
> > > directory.
> > >
> > > Once you have that, you can publish those same artifacts to your internal
> > > repo and modify your project's pom to download the artifacts from your
> > > internal repo.
> > >
> > > That way, everyone in my group is using the same kafka artifacts that I
> > > am building.
> > >
> > >
> > > Alex Gray
> > >
> > >
> > > -----Original Message-----
> > > From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com]
> > > Sent: Monday, May 06, 2013 10:27 PM
> > > To: users@kafka.apache.org
> > > Subject: Re: maven integration in 0.8
> > >
> > > There are a lot of release-oriented tickets still about- here's what I'm
> > > following pertaining to packaging & getting
> > > shipped-
> > >
> > > Maven-
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > Scala 2.10-
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > Deps upgrade-
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > CodaMetrics versioning-
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > >
> > > On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
> > > > Is there a maven repo we can point to, to just depend on the kafka 0.8
> > > > core, and have all the dependencies get pulled in as needed?
> > > >
> > > > For some reason, I had thought this would be available as part of the
> > > > 0.8 release....
> > > >
> > > > Or do I need to manually create a pom.xml for the core, and host it on
> > > > my nexxus instance (as in 0.7.2?).
> > > >
> > > > Thanks,
> > > >
> > > > Jason
Re: maven integration in 0.8
Posted by Jason Rosenberg <jb...@squareup.com>.
Except that it still has the annoying feature of naming the kafka version
after the scala version, which doesn't make sense, e.g.:
~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT/poms ls -l
total 24
-rw-r--r-- 1 jbr jbr 3786 May 7 13:43 kafka_2.8.0.pom
-rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.pom.md5
-rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.pom.sha1
and
~/.ivy2/local/org.apache/kafka_2.8.0/0.8.0-SNAPSHOT ls -l jars
total 4640
-rw-r--r-- 1 jbr jbr 2367457 May 6 15:54 kafka_2.8.0.jar
-rw-r--r-- 1 jbr jbr 32 May 7 13:44 kafka_2.8.0.jar.md5
-rw-r--r-- 1 jbr jbr 40 May 7 13:44 kafka_2.8.0.jar.sha1
I think it was mentioned previously (0.7.x had the same issue) there's an
easy way to fix this (but I don't remember at the moment). Perhaps we
could get this fixed before releasing?
I'd think the maven artifacts should be named 'kafka_0.8.0.pom' or
'kakfa_0.8.0-SNAPSHOT.pom', etc...
Jason
On Tue, May 7, 2013 at 1:48 PM, Jason Rosenberg <jb...@squareup.com> wrote:
> Alex,
>
> Thanks for the tip, this is exactly what I need.
>
> Jason
>
>
> On Tue, May 7, 2013 at 11:58 AM, Gray, Alex <Al...@inin.com> wrote:
>
>> Hi Jason,
>>
>> I don't know if this will help you, but I'm pulling the source code of
>> kafka down to a local machine and calling:
>> ./sbt update
>> /sbt -Dsbt.log.noformat=true publish-local
>> This will publish the generated pom's and jars to your local ~/.ivy2
>> directory.
>>
>> Once you have that, you can publish those same artifacts to your internal
>> repo and modify your project's pom to download the artifacts from your
>> internal repo.
>>
>> That way, everyone in my group is using the same kafka artifacts that I
>> am building.
>>
>>
>> Alex Gray
>>
>>
>> -----Original Message-----
>> From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com]
>> Sent: Monday, May 06, 2013 10:27 PM
>> To: users@kafka.apache.org
>> Subject: Re: maven integration in 0.8
>>
>> There are a lot of release-oriented tickets still about- here's what I'm
>> following pertaining to packaging & getting
>> shipped-
>>
>> Maven-
>>
>> https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> Scala 2.10-
>>
>> https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> Deps upgrade-
>>
>> https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> CodaMetrics versioning-
>>
>> https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>
>> On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
>> > Is there a maven repo we can point to, to just depend on the kafka 0.8
>> > core, and have all the dependencies get pulled in as needed?
>> >
>> > For some reason, I had thought this would be available as part of the
>> > 0.8 release....
>> >
>> > Or do I need to manually create a pom.xml for the core, and host it on
>> > my nexxus instance (as in 0.7.2?).
>> >
>> > Thanks,
>> >
>> > Jason
>>
>>
>>
>
Re: maven integration in 0.8
Posted by Jason Rosenberg <jb...@squareup.com>.
Alex,
Thanks for the tip, this is exactly what I need.
Jason
On Tue, May 7, 2013 at 11:58 AM, Gray, Alex <Al...@inin.com> wrote:
> Hi Jason,
>
> I don't know if this will help you, but I'm pulling the source code of
> kafka down to a local machine and calling:
> ./sbt update
> /sbt -Dsbt.log.noformat=true publish-local
> This will publish the generated pom's and jars to your local ~/.ivy2
> directory.
>
> Once you have that, you can publish those same artifacts to your internal
> repo and modify your project's pom to download the artifacts from your
> internal repo.
>
> That way, everyone in my group is using the same kafka artifacts that I am
> building.
>
>
> Alex Gray
>
>
> -----Original Message-----
> From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com]
> Sent: Monday, May 06, 2013 10:27 PM
> To: users@kafka.apache.org
> Subject: Re: maven integration in 0.8
>
> There are a lot of release-oriented tickets still about- here's what I'm
> following pertaining to packaging & getting
> shipped-
>
> Maven-
>
> https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> Scala 2.10-
>
> https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> Deps upgrade-
>
> https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> CodaMetrics versioning-
>
> https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>
> On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
> > Is there a maven repo we can point to, to just depend on the kafka 0.8
> > core, and have all the dependencies get pulled in as needed?
> >
> > For some reason, I had thought this would be available as part of the
> > 0.8 release....
> >
> > Or do I need to manually create a pom.xml for the core, and host it on
> > my nexxus instance (as in 0.7.2?).
> >
> > Thanks,
> >
> > Jason
>
>
>
RE: maven integration in 0.8
Posted by "Gray, Alex" <Al...@inin.com>.
Hi Jason,
I don't know if this will help you, but I'm pulling the source code of kafka down to a local machine and calling:
./sbt update
/sbt -Dsbt.log.noformat=true publish-local
This will publish the generated pom's and jars to your local ~/.ivy2 directory.
Once you have that, you can publish those same artifacts to your internal repo and modify your project's pom to download the artifacts from your internal repo.
That way, everyone in my group is using the same kafka artifacts that I am building.
Alex Gray
-----Original Message-----
From: rektide@voodoowarez.com [mailto:rektide@voodoowarez.com]
Sent: Monday, May 06, 2013 10:27 PM
To: users@kafka.apache.org
Subject: Re: maven integration in 0.8
There are a lot of release-oriented tickets still about- here's what I'm following pertaining to packaging & getting
shipped-
Maven-
https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
Scala 2.10-
https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
Deps upgrade-
https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
CodaMetrics versioning-
https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
> Is there a maven repo we can point to, to just depend on the kafka 0.8
> core, and have all the dependencies get pulled in as needed?
>
> For some reason, I had thought this would be available as part of the
> 0.8 release....
>
> Or do I need to manually create a pom.xml for the core, and host it on
> my nexxus instance (as in 0.7.2?).
>
> Thanks,
>
> Jason
Re: maven integration in 0.8
Posted by re...@voodoowarez.com.
There are a lot of release-oriented tickets still about-
here's what I'm following pertaining to packaging & getting
shipped-
Maven-
https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
Scala 2.10-
https://issues.apache.org/jira/browse/KAFKA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
Deps upgrade-
https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
CodaMetrics versioning-
https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
On Mon, May 06, 2013 at 04:31:50PM -0700, Jason Rosenberg wrote:
> Is there a maven repo we can point to, to just depend on the kafka 0.8
> core, and have all the dependencies get pulled in as needed?
>
> For some reason, I had thought this would be available as part of the 0.8
> release....
>
> Or do I need to manually create a pom.xml for the core, and host it on my
> nexxus instance (as in 0.7.2?).
>
> Thanks,
>
> Jason
Re: maven integration in 0.8
Posted by Joe Stein <cr...@gmail.com>.
0.8 is not released yet
I need to start catching up on tickets starting here.
On Mon, May 6, 2013 at 7:31 PM, Jason Rosenberg <jb...@squareup.com> wrote:
> Is there a maven repo we can point to, to just depend on the kafka 0.8
> core, and have all the dependencies get pulled in as needed?
>
> For some reason, I had thought this would be available as part of the 0.8
> release....
>
> Or do I need to manually create a pom.xml for the core, and host it on my
> nexxus instance (as in 0.7.2?).
>
> Thanks,
>
> Jason
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/