Thoughts on using Ant+Ivy for the build?

[David Arthur <mu...@gmail.com>]

After getting frustrated with SBT, and being unable to figure out 
seemingly simple problems like KAFKA-843, I decided to try something 
completely different.

I spent some time yesterday adapting some Ant/Ivy boilerplate I use for 
projects to Kafka. It was actually pretty easy to get working (Kafka is 
a very simple build), and I think it's _much_ cleaner than the existing 
SBT stuff.

Attached is a patchset of the work I did. N.B., this is totally 
experimental and only considers the "core" part of the project.

At this point I can:

* Resolve all deps through Ivy (except yammer.metrics which is checked in)
* Resolve different versions of Scala through Ivy (i.e., cross 
compile-ability)
* Compile the source
* Run all the unit tests (all passing)
* Compile a jar
* Package a tarball of the libs, conf, and bin scripts

Since all the libraries are localized to the project (and not in 
~/.ivy2), the packaging is trivial. Also, the bin scripts could be 
cleaned up significantly (which they need to be IMO).

I would love to hear what everyone thinks of this. Am I crazy? Is SBT 
really better? Convince me!

-David

Re: Thoughts on using Ant+Ivy for the build?

[David Arthur <mu...@gmail.com>]

Jun, I have opened a ticket targeting 0.9

https://issues.apache.org/jira/browse/KAFKA-855

-David

On 4/8/13 1:06 AM, Jun Rao wrote:
> David,
>
> We'd be open to a new build tool if SBT is deemed too hard to use. We can
> revisit this post 0.8. Do you want to open a jira to track this?
>
> Thanks,
>
> Jun
>
>
> On Fri, Apr 5, 2013 at 12:55 PM, David Arthur <mu...@gmail.com> wrote:
>
>> After getting frustrated with SBT, and being unable to figure out
>> seemingly simple problems like KAFKA-843, I decided to try something
>> completely different.
>>
>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
>> projects to Kafka. It was actually pretty easy to get working (Kafka is a
>> very simple build), and I think it's _much_ cleaner than the existing SBT
>> stuff.
>>
>> Attached is a patchset of the work I did. N.B., this is totally
>> experimental and only considers the "core" part of the project.
>>
>> At this point I can:
>>
>> * Resolve all deps through Ivy (except yammer.metrics which is checked in)
>> * Resolve different versions of Scala through Ivy (i.e., cross
>> compile-ability)
>> * Compile the source
>> * Run all the unit tests (all passing)
>> * Compile a jar
>> * Package a tarball of the libs, conf, and bin scripts
>>
>> Since all the libraries are localized to the project (and not in ~/.ivy2),
>> the packaging is trivial. Also, the bin scripts could be cleaned up
>> significantly (which they need to be IMO).
>>
>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>> really better? Convince me!
>>
>> -David
>>
>>
>>
>>

Re: Thoughts on using Ant+Ivy for the build?

[Jun Rao <ju...@gmail.com>]

David,

We'd be open to a new build tool if SBT is deemed too hard to use. We can
revisit this post 0.8. Do you want to open a jira to track this?

Thanks,

Jun


On Fri, Apr 5, 2013 at 12:55 PM, David Arthur <mu...@gmail.com> wrote:

> After getting frustrated with SBT, and being unable to figure out
> seemingly simple problems like KAFKA-843, I decided to try something
> completely different.
>
> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
> projects to Kafka. It was actually pretty easy to get working (Kafka is a
> very simple build), and I think it's _much_ cleaner than the existing SBT
> stuff.
>
> Attached is a patchset of the work I did. N.B., this is totally
> experimental and only considers the "core" part of the project.
>
> At this point I can:
>
> * Resolve all deps through Ivy (except yammer.metrics which is checked in)
> * Resolve different versions of Scala through Ivy (i.e., cross
> compile-ability)
> * Compile the source
> * Run all the unit tests (all passing)
> * Compile a jar
> * Package a tarball of the libs, conf, and bin scripts
>
> Since all the libraries are localized to the project (and not in ~/.ivy2),
> the packaging is trivial. Also, the bin scripts could be cleaned up
> significantly (which they need to be IMO).
>
> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
> really better? Convince me!
>
> -David
>
>
>
>

Re: Thoughts on using Ant+Ivy for the build?

[David Arthur <mu...@gmail.com>]

Yes Ant supports it too since scalac supports it.

Most likely Maven and Gradle are using scalac or SBT under the hood 
(unless they have written their own compilers - unlikely). However, from 
what I've read when researching the Ant build, these scalac options 
(-make:transitive -dependencyfile) are going away.

On 4/8/13 5:15 PM, Scott Carey wrote:
> Let me google that for myself:
>
> Gradle has incremental scala compilation:
> http://www.gradle.org/docs/current/userguide/scala_plugin.html
> Maven does too:
> http://davidb.github.io/scala-maven-plugin/example_incremental.html
> So I suspect it could be solved with Ant.
>
> Maven and Gradle both support calling ant tasks from within them, if
> necessary.
>
>
> In general, any tool will work if maintained and used properly, and any
> tool can be abused and poorly organized for maintenance.
>
> On 4/8/13 2:09 PM, "Scott Carey" <sc...@richrelevance.com> wrote:
>
>> Gradle is also gaining a lot of popularity, and has a good deal of Scala
>> support.  I don't know if it has incremental compilation.  The maven scala
>> plugin at one time had incremental compilation, I don't know what state
>> that is in now.
>>
>> I'm giving learning enough SBT a shot right now to clean the current mess
>> up, and will submit a patch as part of Kafka-854.  Having KAFKA-826
>> committed would help.
>>
>> On 4/8/13 1:31 PM, "David Arthur" <mu...@gmail.com> wrote:
>>
>>> Targeting multiple Scala versions is easy with Ant/Ivy and I believe is
>>> possible in Maven. Incremental compilation is available through some
>>> IDEs:
>>>
>>> Eclipse has a plugin from Typesafe that does incremental build:
>>> http://typesafe.com/technology/scala-ide
>>> Intellij supports it by using SBT:
>>> http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/
>>>
>>> Lowly vim users like myself would have to suffer longer compile times
>>>
>>> -David
>>>
>>> On 4/8/13 4:20 PM, Jay Kreps wrote:
>>>> I think the biggest problem is that there isn't clear ownership of the
>>>> build stuff we have. SBT is probably fine if you know it well. Ant and
>>>> maven are probably the same. The problem is that none of the
>>>> committers seem to know SBT well that that is a big barrier for
>>>> getting basic stuff like a good unit test report, etc.
>>>>
>>>> My personal preference would be for Maven followed by Ant followed by
>>>> anything else (including SBT), but I think the ownership maintenance
>>>> issue is more important than the framework.
>>>>
>>>> The two useful things that SBT bought us where compile targets (i.e.
>>>> scala 2.8 vs 2.9) and incremental compilation. I think both are
>>>> available outside sbt now...?
>>>>
>>>> -Jay
>>>>
>>>>
>>>> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur <mu...@gmail.com> wrote:
>>>>> Ant and Ivy also bring in lots of free stuff as well as the
>>>>> flexibility to
>>>>> do non-standard things
>>>>>
>>>>> It seems that SBT is a moving target (as is Scala itself, being such a
>>>>> new
>>>>> tech), and keeping up with SBT changes and people's plugins hosted on
>>>>> GitHub
>>>>> sounds like a pain. Maven and Ant/Ivy are mature and stable and well
>>>>> understood.
>>>>>
>>>>> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is
>>>>> clear
>>>>> that there are issues with the current build. I'm proposing Ant+Ivy,
>>>>> if
>>>>> enough people buy into this - great we'll use Ant. If not, also great
>>>>> - I'm
>>>>> simply forcing the dialog :)
>>>>>
>>>>> Also, if what I'm proposing is accepted, I would be willing to
>>>>> maintain this
>>>>> piece of the project in the longer term.
>>>>>
>>>>> -David
>>>>>
>>>>>
>>>>> On 4/8/13 3:40 PM, Scott Carey wrote:
>>>>>> My first reaction to looking at the current build was that I could do
>>>>>> a
>>>>>> lot better with Maven and I'd get a lot of free stuff with that
>>>>>> (dependency:tree, versions:display-dependency-updates), except that
>>>>>> cross
>>>>>> scala versions would be uglier.  Modularization is trivial with maven
>>>>>> (the
>>>>>> stuff outside core).
>>>>>>
>>>>>> A little more digging, and it seems that the current sbt build could
>>>>>> be
>>>>>> significantly simplified and has a lot of cruft.
>>>>>>
>>>>>>
>>>>>> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>>>>>>
>>>>>>> After getting frustrated with SBT, and being unable to figure out
>>>>>>> seemingly simple problems like KAFKA-843, I decided to try something
>>>>>>> completely different.
>>>>>>>
>>>>>>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use
>>>>>>> for
>>>>>>> projects to Kafka. It was actually pretty easy to get working (Kafka
>>>>>>> is
>>>>>>> a very simple build), and I think it's _much_ cleaner than the
>>>>>>> existing
>>>>>>> SBT stuff.
>>>>>>>
>>>>>>> Attached is a patchset of the work I did. N.B., this is totally
>>>>>>> experimental and only considers the "core" part of the project.
>>>>>>>
>>>>>>> At this point I can:
>>>>>>>
>>>>>>> * Resolve all deps through Ivy (except yammer.metrics which is
>>>>>>> checked
>>>>>>> in)
>>>>>>> * Resolve different versions of Scala through Ivy (i.e., cross
>>>>>>> compile-ability)
>>>>>>> * Compile the source
>>>>>>> * Run all the unit tests (all passing)
>>>>>>> * Compile a jar
>>>>>>> * Package a tarball of the libs, conf, and bin scripts
>>>>>>>
>>>>>>> Since all the libraries are localized to the project (and not in
>>>>>>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>>>>>>> cleaned up significantly (which they need to be IMO).
>>>>>>>
>>>>>>> I would love to hear what everyone thinks of this. Am I crazy? Is
>>>>>>> SBT
>>>>>>> really better? Convince me!
>>>>>>>
>>>>>>> -David
>>>>>>>
>>>>>>>
>>>>>>>

Re: Thoughts on using Ant+Ivy for the build?

[Scott Carey <sc...@richrelevance.com>]

Let me google that for myself:

Gradle has incremental scala compilation:
http://www.gradle.org/docs/current/userguide/scala_plugin.html
Maven does too: 
http://davidb.github.io/scala-maven-plugin/example_incremental.html
So I suspect it could be solved with Ant.

Maven and Gradle both support calling ant tasks from within them, if
necessary.


In general, any tool will work if maintained and used properly, and any
tool can be abused and poorly organized for maintenance.

On 4/8/13 2:09 PM, "Scott Carey" <sc...@richrelevance.com> wrote:

>Gradle is also gaining a lot of popularity, and has a good deal of Scala
>support.  I don't know if it has incremental compilation.  The maven scala
>plugin at one time had incremental compilation, I don't know what state
>that is in now.
>
>I'm giving learning enough SBT a shot right now to clean the current mess
>up, and will submit a patch as part of Kafka-854.  Having KAFKA-826
>committed would help.
>
>On 4/8/13 1:31 PM, "David Arthur" <mu...@gmail.com> wrote:
>
>>Targeting multiple Scala versions is easy with Ant/Ivy and I believe is
>>possible in Maven. Incremental compilation is available through some
>>IDEs:
>>
>>Eclipse has a plugin from Typesafe that does incremental build:
>>http://typesafe.com/technology/scala-ide
>>Intellij supports it by using SBT:
>>http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/
>>
>>Lowly vim users like myself would have to suffer longer compile times
>>
>>-David
>>
>>On 4/8/13 4:20 PM, Jay Kreps wrote:
>>> I think the biggest problem is that there isn't clear ownership of the
>>> build stuff we have. SBT is probably fine if you know it well. Ant and
>>> maven are probably the same. The problem is that none of the
>>> committers seem to know SBT well that that is a big barrier for
>>> getting basic stuff like a good unit test report, etc.
>>>
>>> My personal preference would be for Maven followed by Ant followed by
>>> anything else (including SBT), but I think the ownership maintenance
>>> issue is more important than the framework.
>>>
>>> The two useful things that SBT bought us where compile targets (i.e.
>>> scala 2.8 vs 2.9) and incremental compilation. I think both are
>>> available outside sbt now...?
>>>
>>> -Jay
>>>
>>>
>>> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur <mu...@gmail.com> wrote:
>>>> Ant and Ivy also bring in lots of free stuff as well as the
>>>>flexibility to
>>>> do non-standard things
>>>>
>>>> It seems that SBT is a moving target (as is Scala itself, being such a
>>>>new
>>>> tech), and keeping up with SBT changes and people's plugins hosted on
>>>>GitHub
>>>> sounds like a pain. Maven and Ant/Ivy are mature and stable and well
>>>> understood.
>>>>
>>>> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is
>>>>clear
>>>> that there are issues with the current build. I'm proposing Ant+Ivy,
>>>>if
>>>> enough people buy into this - great we'll use Ant. If not, also great
>>>>- I'm
>>>> simply forcing the dialog :)
>>>>
>>>> Also, if what I'm proposing is accepted, I would be willing to
>>>>maintain this
>>>> piece of the project in the longer term.
>>>>
>>>> -David
>>>>
>>>>
>>>> On 4/8/13 3:40 PM, Scott Carey wrote:
>>>>> My first reaction to looking at the current build was that I could do
>>>>>a
>>>>> lot better with Maven and I'd get a lot of free stuff with that
>>>>> (dependency:tree, versions:display-dependency-updates), except that
>>>>>cross
>>>>> scala versions would be uglier.  Modularization is trivial with maven
>>>>>(the
>>>>> stuff outside core).
>>>>>
>>>>> A little more digging, and it seems that the current sbt build could
>>>>>be
>>>>> significantly simplified and has a lot of cruft.
>>>>>
>>>>>
>>>>> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>>>>>
>>>>>> After getting frustrated with SBT, and being unable to figure out
>>>>>> seemingly simple problems like KAFKA-843, I decided to try something
>>>>>> completely different.
>>>>>>
>>>>>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use
>>>>>>for
>>>>>> projects to Kafka. It was actually pretty easy to get working (Kafka
>>>>>>is
>>>>>> a very simple build), and I think it's _much_ cleaner than the
>>>>>>existing
>>>>>> SBT stuff.
>>>>>>
>>>>>> Attached is a patchset of the work I did. N.B., this is totally
>>>>>> experimental and only considers the "core" part of the project.
>>>>>>
>>>>>> At this point I can:
>>>>>>
>>>>>> * Resolve all deps through Ivy (except yammer.metrics which is
>>>>>>checked
>>>>>> in)
>>>>>> * Resolve different versions of Scala through Ivy (i.e., cross
>>>>>> compile-ability)
>>>>>> * Compile the source
>>>>>> * Run all the unit tests (all passing)
>>>>>> * Compile a jar
>>>>>> * Package a tarball of the libs, conf, and bin scripts
>>>>>>
>>>>>> Since all the libraries are localized to the project (and not in
>>>>>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>>>>>> cleaned up significantly (which they need to be IMO).
>>>>>>
>>>>>> I would love to hear what everyone thinks of this. Am I crazy? Is
>>>>>>SBT
>>>>>> really better? Convince me!
>>>>>>
>>>>>> -David
>>>>>>
>>>>>>
>>>>>>
>>
>

Re: Thoughts on using Ant+Ivy for the build?

[Scott Carey <sc...@richrelevance.com>]

Gradle is also gaining a lot of popularity, and has a good deal of Scala
support.  I don't know if it has incremental compilation.  The maven scala
plugin at one time had incremental compilation, I don't know what state
that is in now.

I'm giving learning enough SBT a shot right now to clean the current mess
up, and will submit a patch as part of Kafka-854.  Having KAFKA-826
committed would help.

On 4/8/13 1:31 PM, "David Arthur" <mu...@gmail.com> wrote:

>Targeting multiple Scala versions is easy with Ant/Ivy and I believe is
>possible in Maven. Incremental compilation is available through some IDEs:
>
>Eclipse has a plugin from Typesafe that does incremental build:
>http://typesafe.com/technology/scala-ide
>Intellij supports it by using SBT:
>http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/
>
>Lowly vim users like myself would have to suffer longer compile times
>
>-David
>
>On 4/8/13 4:20 PM, Jay Kreps wrote:
>> I think the biggest problem is that there isn't clear ownership of the
>> build stuff we have. SBT is probably fine if you know it well. Ant and
>> maven are probably the same. The problem is that none of the
>> committers seem to know SBT well that that is a big barrier for
>> getting basic stuff like a good unit test report, etc.
>>
>> My personal preference would be for Maven followed by Ant followed by
>> anything else (including SBT), but I think the ownership maintenance
>> issue is more important than the framework.
>>
>> The two useful things that SBT bought us where compile targets (i.e.
>> scala 2.8 vs 2.9) and incremental compilation. I think both are
>> available outside sbt now...?
>>
>> -Jay
>>
>>
>> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur <mu...@gmail.com> wrote:
>>> Ant and Ivy also bring in lots of free stuff as well as the
>>>flexibility to
>>> do non-standard things
>>>
>>> It seems that SBT is a moving target (as is Scala itself, being such a
>>>new
>>> tech), and keeping up with SBT changes and people's plugins hosted on
>>>GitHub
>>> sounds like a pain. Maven and Ant/Ivy are mature and stable and well
>>> understood.
>>>
>>> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is
>>>clear
>>> that there are issues with the current build. I'm proposing Ant+Ivy, if
>>> enough people buy into this - great we'll use Ant. If not, also great
>>>- I'm
>>> simply forcing the dialog :)
>>>
>>> Also, if what I'm proposing is accepted, I would be willing to
>>>maintain this
>>> piece of the project in the longer term.
>>>
>>> -David
>>>
>>>
>>> On 4/8/13 3:40 PM, Scott Carey wrote:
>>>> My first reaction to looking at the current build was that I could do
>>>>a
>>>> lot better with Maven and I'd get a lot of free stuff with that
>>>> (dependency:tree, versions:display-dependency-updates), except that
>>>>cross
>>>> scala versions would be uglier.  Modularization is trivial with maven
>>>>(the
>>>> stuff outside core).
>>>>
>>>> A little more digging, and it seems that the current sbt build could
>>>>be
>>>> significantly simplified and has a lot of cruft.
>>>>
>>>>
>>>> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>>>>
>>>>> After getting frustrated with SBT, and being unable to figure out
>>>>> seemingly simple problems like KAFKA-843, I decided to try something
>>>>> completely different.
>>>>>
>>>>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use
>>>>>for
>>>>> projects to Kafka. It was actually pretty easy to get working (Kafka
>>>>>is
>>>>> a very simple build), and I think it's _much_ cleaner than the
>>>>>existing
>>>>> SBT stuff.
>>>>>
>>>>> Attached is a patchset of the work I did. N.B., this is totally
>>>>> experimental and only considers the "core" part of the project.
>>>>>
>>>>> At this point I can:
>>>>>
>>>>> * Resolve all deps through Ivy (except yammer.metrics which is
>>>>>checked
>>>>> in)
>>>>> * Resolve different versions of Scala through Ivy (i.e., cross
>>>>> compile-ability)
>>>>> * Compile the source
>>>>> * Run all the unit tests (all passing)
>>>>> * Compile a jar
>>>>> * Package a tarball of the libs, conf, and bin scripts
>>>>>
>>>>> Since all the libraries are localized to the project (and not in
>>>>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>>>>> cleaned up significantly (which they need to be IMO).
>>>>>
>>>>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>>>>> really better? Convince me!
>>>>>
>>>>> -David
>>>>>
>>>>>
>>>>>
>

Re: Thoughts on using Ant+Ivy for the build?

[David Arthur <mu...@gmail.com>]

Targeting multiple Scala versions is easy with Ant/Ivy and I believe is 
possible in Maven. Incremental compilation is available through some IDEs:

Eclipse has a plugin from Typesafe that does incremental build: 
http://typesafe.com/technology/scala-ide
Intellij supports it by using SBT: 
http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/

Lowly vim users like myself would have to suffer longer compile times

-David

On 4/8/13 4:20 PM, Jay Kreps wrote:
> I think the biggest problem is that there isn't clear ownership of the
> build stuff we have. SBT is probably fine if you know it well. Ant and
> maven are probably the same. The problem is that none of the
> committers seem to know SBT well that that is a big barrier for
> getting basic stuff like a good unit test report, etc.
>
> My personal preference would be for Maven followed by Ant followed by
> anything else (including SBT), but I think the ownership maintenance
> issue is more important than the framework.
>
> The two useful things that SBT bought us where compile targets (i.e.
> scala 2.8 vs 2.9) and incremental compilation. I think both are
> available outside sbt now...?
>
> -Jay
>
>
> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur <mu...@gmail.com> wrote:
>> Ant and Ivy also bring in lots of free stuff as well as the flexibility to
>> do non-standard things
>>
>> It seems that SBT is a moving target (as is Scala itself, being such a new
>> tech), and keeping up with SBT changes and people's plugins hosted on GitHub
>> sounds like a pain. Maven and Ant/Ivy are mature and stable and well
>> understood.
>>
>> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear
>> that there are issues with the current build. I'm proposing Ant+Ivy, if
>> enough people buy into this - great we'll use Ant. If not, also great - I'm
>> simply forcing the dialog :)
>>
>> Also, if what I'm proposing is accepted, I would be willing to maintain this
>> piece of the project in the longer term.
>>
>> -David
>>
>>
>> On 4/8/13 3:40 PM, Scott Carey wrote:
>>> My first reaction to looking at the current build was that I could do a
>>> lot better with Maven and I'd get a lot of free stuff with that
>>> (dependency:tree, versions:display-dependency-updates), except that cross
>>> scala versions would be uglier.  Modularization is trivial with maven (the
>>> stuff outside core).
>>>
>>> A little more digging, and it seems that the current sbt build could be
>>> significantly simplified and has a lot of cruft.
>>>
>>>
>>> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>>>
>>>> After getting frustrated with SBT, and being unable to figure out
>>>> seemingly simple problems like KAFKA-843, I decided to try something
>>>> completely different.
>>>>
>>>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
>>>> projects to Kafka. It was actually pretty easy to get working (Kafka is
>>>> a very simple build), and I think it's _much_ cleaner than the existing
>>>> SBT stuff.
>>>>
>>>> Attached is a patchset of the work I did. N.B., this is totally
>>>> experimental and only considers the "core" part of the project.
>>>>
>>>> At this point I can:
>>>>
>>>> * Resolve all deps through Ivy (except yammer.metrics which is checked
>>>> in)
>>>> * Resolve different versions of Scala through Ivy (i.e., cross
>>>> compile-ability)
>>>> * Compile the source
>>>> * Run all the unit tests (all passing)
>>>> * Compile a jar
>>>> * Package a tarball of the libs, conf, and bin scripts
>>>>
>>>> Since all the libraries are localized to the project (and not in
>>>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>>>> cleaned up significantly (which they need to be IMO).
>>>>
>>>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>>>> really better? Convince me!
>>>>
>>>> -David
>>>>
>>>>
>>>>

Re: Thoughts on using Ant+Ivy for the build?

[Jay Kreps <ja...@gmail.com>]

I think the biggest problem is that there isn't clear ownership of the
build stuff we have. SBT is probably fine if you know it well. Ant and
maven are probably the same. The problem is that none of the
committers seem to know SBT well that that is a big barrier for
getting basic stuff like a good unit test report, etc.

My personal preference would be for Maven followed by Ant followed by
anything else (including SBT), but I think the ownership maintenance
issue is more important than the framework.

The two useful things that SBT bought us where compile targets (i.e.
scala 2.8 vs 2.9) and incremental compilation. I think both are
available outside sbt now...?

-Jay


On Mon, Apr 8, 2013 at 12:52 PM, David Arthur <mu...@gmail.com> wrote:
> Ant and Ivy also bring in lots of free stuff as well as the flexibility to
> do non-standard things
>
> It seems that SBT is a moving target (as is Scala itself, being such a new
> tech), and keeping up with SBT changes and people's plugins hosted on GitHub
> sounds like a pain. Maven and Ant/Ivy are mature and stable and well
> understood.
>
> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear
> that there are issues with the current build. I'm proposing Ant+Ivy, if
> enough people buy into this - great we'll use Ant. If not, also great - I'm
> simply forcing the dialog :)
>
> Also, if what I'm proposing is accepted, I would be willing to maintain this
> piece of the project in the longer term.
>
> -David
>
>
> On 4/8/13 3:40 PM, Scott Carey wrote:
>>
>> My first reaction to looking at the current build was that I could do a
>> lot better with Maven and I'd get a lot of free stuff with that
>> (dependency:tree, versions:display-dependency-updates), except that cross
>> scala versions would be uglier.  Modularization is trivial with maven (the
>> stuff outside core).
>>
>> A little more digging, and it seems that the current sbt build could be
>> significantly simplified and has a lot of cruft.
>>
>>
>> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>>
>>> After getting frustrated with SBT, and being unable to figure out
>>> seemingly simple problems like KAFKA-843, I decided to try something
>>> completely different.
>>>
>>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
>>> projects to Kafka. It was actually pretty easy to get working (Kafka is
>>> a very simple build), and I think it's _much_ cleaner than the existing
>>> SBT stuff.
>>>
>>> Attached is a patchset of the work I did. N.B., this is totally
>>> experimental and only considers the "core" part of the project.
>>>
>>> At this point I can:
>>>
>>> * Resolve all deps through Ivy (except yammer.metrics which is checked
>>> in)
>>> * Resolve different versions of Scala through Ivy (i.e., cross
>>> compile-ability)
>>> * Compile the source
>>> * Run all the unit tests (all passing)
>>> * Compile a jar
>>> * Package a tarball of the libs, conf, and bin scripts
>>>
>>> Since all the libraries are localized to the project (and not in
>>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>>> cleaned up significantly (which they need to be IMO).
>>>
>>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>>> really better? Convince me!
>>>
>>> -David
>>>
>>>
>>>
>

Re: Thoughts on using Ant+Ivy for the build?

[David Arthur <mu...@gmail.com>]

Ant and Ivy also bring in lots of free stuff as well as the flexibility 
to do non-standard things

It seems that SBT is a moving target (as is Scala itself, being such a 
new tech), and keeping up with SBT changes and people's plugins hosted 
on GitHub sounds like a pain. Maven and Ant/Ivy are mature and stable 
and well understood.

Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear 
that there are issues with the current build. I'm proposing Ant+Ivy, if 
enough people buy into this - great we'll use Ant. If not, also great - 
I'm simply forcing the dialog :)

Also, if what I'm proposing is accepted, I would be willing to maintain 
this piece of the project in the longer term.

-David

On 4/8/13 3:40 PM, Scott Carey wrote:
> My first reaction to looking at the current build was that I could do a
> lot better with Maven and I'd get a lot of free stuff with that
> (dependency:tree, versions:display-dependency-updates), except that cross
> scala versions would be uglier.  Modularization is trivial with maven (the
> stuff outside core).
>
> A little more digging, and it seems that the current sbt build could be
> significantly simplified and has a lot of cruft.
>
>
> On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:
>
>> After getting frustrated with SBT, and being unable to figure out
>> seemingly simple problems like KAFKA-843, I decided to try something
>> completely different.
>>
>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
>> projects to Kafka. It was actually pretty easy to get working (Kafka is
>> a very simple build), and I think it's _much_ cleaner than the existing
>> SBT stuff.
>>
>> Attached is a patchset of the work I did. N.B., this is totally
>> experimental and only considers the "core" part of the project.
>>
>> At this point I can:
>>
>> * Resolve all deps through Ivy (except yammer.metrics which is checked in)
>> * Resolve different versions of Scala through Ivy (i.e., cross
>> compile-ability)
>> * Compile the source
>> * Run all the unit tests (all passing)
>> * Compile a jar
>> * Package a tarball of the libs, conf, and bin scripts
>>
>> Since all the libraries are localized to the project (and not in
>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>> cleaned up significantly (which they need to be IMO).
>>
>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>> really better? Convince me!
>>
>> -David
>>
>>
>>

Re: Thoughts on using Ant+Ivy for the build?

[Scott Carey <sc...@richrelevance.com>]

My first reaction to looking at the current build was that I could do a
lot better with Maven and I'd get a lot of free stuff with that
(dependency:tree, versions:display-dependency-updates), except that cross
scala versions would be uglier.  Modularization is trivial with maven (the
stuff outside core).

A little more digging, and it seems that the current sbt build could be
significantly simplified and has a lot of cruft.


On 4/5/13 12:55 PM, "David Arthur" <mu...@gmail.com> wrote:

>After getting frustrated with SBT, and being unable to figure out
>seemingly simple problems like KAFKA-843, I decided to try something
>completely different.
>
>I spent some time yesterday adapting some Ant/Ivy boilerplate I use for
>projects to Kafka. It was actually pretty easy to get working (Kafka is
>a very simple build), and I think it's _much_ cleaner than the existing
>SBT stuff.
>
>Attached is a patchset of the work I did. N.B., this is totally
>experimental and only considers the "core" part of the project.
>
>At this point I can:
>
>* Resolve all deps through Ivy (except yammer.metrics which is checked in)
>* Resolve different versions of Scala through Ivy (i.e., cross
>compile-ability)
>* Compile the source
>* Run all the unit tests (all passing)
>* Compile a jar
>* Package a tarball of the libs, conf, and bin scripts
>
>Since all the libraries are localized to the project (and not in
>~/.ivy2), the packaging is trivial. Also, the bin scripts could be
>cleaned up significantly (which they need to be IMO).
>
>I would love to hear what everyone thinks of this. Am I crazy? Is SBT
>really better? Convince me!
>
>-David
>
>
>