how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

hey we used to publish spark inhouse by simply overriding the publishTo
setting. but now that we are integrated in SBT with maven i cannot find it
anymore.

i tried looking into the pom file, but after reading 1144 lines of xml i
1) havent found anything that looks like publishing
2) i feel somewhat sick too
3) i am considering alternative careers to developing...

where am i supposed to look?
thanks for your help!

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

i just looked at my dependencies in sbt, and when using cdh4.5.0
dependencies i see that hadoop clients pulls in jboss netty (via zookeeper)
and asm 3.x (via jersey-server). so somehow these exclusion rules are not
working anymore? i will look into sbt-pom-reader a bit to try to understand
whats happening


On Mon, Jul 28, 2014 at 8:45 PM, Patrick Wendell <pw...@gmail.com> wrote:

> All of the scripts we use to publish Spark releases are in the Spark
> repo itself, so you could follow these as a guideline. The publishing
> process in Maven is similar to in SBT:
>
>
> https://github.com/apache/spark/blob/master/dev/create-release/create-release.sh#L65
>
> On Mon, Jul 28, 2014 at 12:39 PM, Koert Kuipers <ko...@tresata.com> wrote:
> > ah ok thanks. guess i am gonna read up about maven-release-plugin then!
> >
> >
> > On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
> >>
> >> This is not something you edit yourself. The Maven release plugin
> >> manages setting all this. I think virtually everything you're worried
> >> about is done for you by this plugin.
> >>
> >> Maven requires artifacts to set a version and it can't inherit one. I
> >> feel like I understood the reason this is necessary at one point.
> >>
> >> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >> > and if i want to change the version, it seems i have to change it in
> all
> >> > 23
> >> > pom files? mhhh. is it mandatory for these sub-project pom files to
> >> > repeat
> >> > that version info? useful?
> >> >
> >> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
> >> > 23
> >> >
> >> >
> >> >
> >> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
> >> > wrote:
> >> >>
> >> >> hey we used to publish spark inhouse by simply overriding the
> publishTo
> >> >> setting. but now that we are integrated in SBT with maven i cannot
> find
> >> >> it
> >> >> anymore.
> >> >>
> >> >> i tried looking into the pom file, but after reading 1144 lines of
> xml
> >> >> i
> >> >> 1) havent found anything that looks like publishing
> >> >> 2) i feel somewhat sick too
> >> >> 3) i am considering alternative careers to developing...
> >> >>
> >> >> where am i supposed to look?
> >> >> thanks for your help!
> >> >
> >> >
> >
> >
>

Re: how to publish spark inhouse?

[Patrick Wendell <pw...@gmail.com>]

All of the scripts we use to publish Spark releases are in the Spark
repo itself, so you could follow these as a guideline. The publishing
process in Maven is similar to in SBT:

https://github.com/apache/spark/blob/master/dev/create-release/create-release.sh#L65

On Mon, Jul 28, 2014 at 12:39 PM, Koert Kuipers <ko...@tresata.com> wrote:
> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
>
>
> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
>>
>> This is not something you edit yourself. The Maven release plugin
>> manages setting all this. I think virtually everything you're worried
>> about is done for you by this plugin.
>>
>> Maven requires artifacts to set a version and it can't inherit one. I
>> feel like I understood the reason this is necessary at one point.
>>
>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com> wrote:
>> > and if i want to change the version, it seems i have to change it in all
>> > 23
>> > pom files? mhhh. is it mandatory for these sub-project pom files to
>> > repeat
>> > that version info? useful?
>> >
>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
>> > 23
>> >
>> >
>> >
>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
>> > wrote:
>> >>
>> >> hey we used to publish spark inhouse by simply overriding the publishTo
>> >> setting. but now that we are integrated in SBT with maven i cannot find
>> >> it
>> >> anymore.
>> >>
>> >> i tried looking into the pom file, but after reading 1144 lines of xml
>> >> i
>> >> 1) havent found anything that looks like publishing
>> >> 2) i feel somewhat sick too
>> >> 3) i am considering alternative careers to developing...
>> >>
>> >> where am i supposed to look?
>> >> thanks for your help!
>> >
>> >
>
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

ah there is an apache parent that i need to go look at. thats why i can't
find so many things. thanks


On Tue, Jul 29, 2014 at 4:08 PM, Sean Owen <so...@cloudera.com> wrote:

> Regarding SBT: I don't think SBT has equivalents for some Maven
> concepts like "runtime" scope. The resulting assembly-like JARs might
> not be what's intended. I think that's part of why Maven is the build
> of reference. You might be able to get the SBT build to do the right
> thing with fiddling. It was my understanding that sbt-pom-reader
> didn't quite support everything required to make the same artifact. So
> I think SBT is not suitable for making release-like things, even for
> an internal snapshot.
>
> deploy is the simplest goal for publishing things like snapshots, yes.
> It just makes artifacts and pushes them, without going through a lot
> of release rigmarole like setting up versions, tests, tagging,
> committing, etc. I use it for simple snapshots.
>
> mvn install is the even simpler thing, as it deploys your artifacts to
> your local machine repo, where you can depend on them locally.
> Neither of those two update all the versions for you AFAIK. Sorry.
> It's usually for snapshots, where the version is already set to the
> right snapshot version by default.
>
> Getting to know the Maven plugins takes some work, but eventually you
> save more time and errors using them than doing it manually.
>
> distributionManagement is inherited from the standard Apache parent.
>
> On Tue, Jul 29, 2014 at 8:58 PM, Koert Kuipers <ko...@tresata.com> wrote:
> > all i want to do is 1) change the version number and 2) publish spark to
> our
> > internal maven repo.
> >
> > i just learned that the maven way what i want to do is a "deploy", not a
> > "release" (neither of which makes sense to me.. how about publish? but
> ok).
> > although me wanting to change a version number is apparently too much to
> > ask, without using a plugin (the maven-release-plugin?). thats fine. i
> will
> > ignore that and go change all the files by hand.
> >
> > so i now understand i need to do a mvn deploy, and there should be a
> > distributionManagement section somewhere in the pom. ok great i can try
> > adding that myself. but how come it isnt there by default? i mean, spark
> > does end up on mavem somehow?
> >
> >
> >
> >
> > On Mon, Jul 28, 2014 at 3:39 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >>
> >> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
> >>
> >>
> >> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
> >>>
> >>> This is not something you edit yourself. The Maven release plugin
> >>> manages setting all this. I think virtually everything you're worried
> >>> about is done for you by this plugin.
> >>>
> >>> Maven requires artifacts to set a version and it can't inherit one. I
> >>> feel like I understood the reason this is necessary at one point.
> >>>
> >>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >>> > and if i want to change the version, it seems i have to change it in
> >>> > all 23
> >>> > pom files? mhhh. is it mandatory for these sub-project pom files to
> >>> > repeat
> >>> > that version info? useful?
> >>> >
> >>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
> >>> > 23
> >>> >
> >>> >
> >>> >
> >>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
> >>> > wrote:
> >>> >>
> >>> >> hey we used to publish spark inhouse by simply overriding the
> >>> >> publishTo
> >>> >> setting. but now that we are integrated in SBT with maven i cannot
> >>> >> find it
> >>> >> anymore.
> >>> >>
> >>> >> i tried looking into the pom file, but after reading 1144 lines of
> xml
> >>> >> i
> >>> >> 1) havent found anything that looks like publishing
> >>> >> 2) i feel somewhat sick too
> >>> >> 3) i am considering alternative careers to developing...
> >>> >>
> >>> >> where am i supposed to look?
> >>> >> thanks for your help!
> >>> >
> >>> >
> >>
> >>
> >
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

ok on the positive side i managed to publish inhouse snapshots with maven.

on the negative side with the exclusions in dependencies getting lost when
the pom gets read into sbt i do not think sbt build is currently usable.
well, at least not for building distributions, i tried that and jboss netty
makes it way into the dependencies which causes the fun exception when you
run a spark job:
*org*.*jboss*.*netty*.*channel*.*ChannelException*: *Failed to bind to ...*


On Tue, Jul 29, 2014 at 4:17 PM, Koert Kuipers <ko...@tresata.com> wrote:

> SBT does actually have a runtime scrope, but things might indeed not
> translate perfectly.
>
> however exclusions being ignored by the sbt-pom-reader is a serious issue.
> not yet sure how to deal with that one
>
>
> On Tue, Jul 29, 2014 at 4:08 PM, Sean Owen <so...@cloudera.com> wrote:
>
>> Regarding SBT: I don't think SBT has equivalents for some Maven
>> concepts like "runtime" scope. The resulting assembly-like JARs might
>> not be what's intended. I think that's part of why Maven is the build
>> of reference. You might be able to get the SBT build to do the right
>> thing with fiddling. It was my understanding that sbt-pom-reader
>> didn't quite support everything required to make the same artifact. So
>> I think SBT is not suitable for making release-like things, even for
>> an internal snapshot.
>>
>> deploy is the simplest goal for publishing things like snapshots, yes.
>> It just makes artifacts and pushes them, without going through a lot
>> of release rigmarole like setting up versions, tests, tagging,
>> committing, etc. I use it for simple snapshots.
>>
>> mvn install is the even simpler thing, as it deploys your artifacts to
>> your local machine repo, where you can depend on them locally.
>> Neither of those two update all the versions for you AFAIK. Sorry.
>> It's usually for snapshots, where the version is already set to the
>> right snapshot version by default.
>>
>> Getting to know the Maven plugins takes some work, but eventually you
>> save more time and errors using them than doing it manually.
>>
>> distributionManagement is inherited from the standard Apache parent.
>>
>> On Tue, Jul 29, 2014 at 8:58 PM, Koert Kuipers <ko...@tresata.com> wrote:
>> > all i want to do is 1) change the version number and 2) publish spark
>> to our
>> > internal maven repo.
>> >
>> > i just learned that the maven way what i want to do is a "deploy", not a
>> > "release" (neither of which makes sense to me.. how about publish? but
>> ok).
>> > although me wanting to change a version number is apparently too much to
>> > ask, without using a plugin (the maven-release-plugin?). thats fine. i
>> will
>> > ignore that and go change all the files by hand.
>> >
>> > so i now understand i need to do a mvn deploy, and there should be a
>> > distributionManagement section somewhere in the pom. ok great i can try
>> > adding that myself. but how come it isnt there by default? i mean, spark
>> > does end up on mavem somehow?
>> >
>> >
>> >
>> >
>> > On Mon, Jul 28, 2014 at 3:39 PM, Koert Kuipers <ko...@tresata.com>
>> wrote:
>> >>
>> >> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
>> >>
>> >>
>> >> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
>> >>>
>> >>> This is not something you edit yourself. The Maven release plugin
>> >>> manages setting all this. I think virtually everything you're worried
>> >>> about is done for you by this plugin.
>> >>>
>> >>> Maven requires artifacts to set a version and it can't inherit one. I
>> >>> feel like I understood the reason this is necessary at one point.
>> >>>
>> >>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com>
>> wrote:
>> >>> > and if i want to change the version, it seems i have to change it in
>> >>> > all 23
>> >>> > pom files? mhhh. is it mandatory for these sub-project pom files to
>> >>> > repeat
>> >>> > that version info? useful?
>> >>> >
>> >>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
>> >>> > 23
>> >>> >
>> >>> >
>> >>> >
>> >>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> hey we used to publish spark inhouse by simply overriding the
>> >>> >> publishTo
>> >>> >> setting. but now that we are integrated in SBT with maven i cannot
>> >>> >> find it
>> >>> >> anymore.
>> >>> >>
>> >>> >> i tried looking into the pom file, but after reading 1144 lines of
>> xml
>> >>> >> i
>> >>> >> 1) havent found anything that looks like publishing
>> >>> >> 2) i feel somewhat sick too
>> >>> >> 3) i am considering alternative careers to developing...
>> >>> >>
>> >>> >> where am i supposed to look?
>> >>> >> thanks for your help!
>> >>> >
>> >>> >
>> >>
>> >>
>> >
>>
>
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

SBT does actually have a runtime scrope, but things might indeed not
translate perfectly.

however exclusions being ignored by the sbt-pom-reader is a serious issue.
not yet sure how to deal with that one


On Tue, Jul 29, 2014 at 4:08 PM, Sean Owen <so...@cloudera.com> wrote:

> Regarding SBT: I don't think SBT has equivalents for some Maven
> concepts like "runtime" scope. The resulting assembly-like JARs might
> not be what's intended. I think that's part of why Maven is the build
> of reference. You might be able to get the SBT build to do the right
> thing with fiddling. It was my understanding that sbt-pom-reader
> didn't quite support everything required to make the same artifact. So
> I think SBT is not suitable for making release-like things, even for
> an internal snapshot.
>
> deploy is the simplest goal for publishing things like snapshots, yes.
> It just makes artifacts and pushes them, without going through a lot
> of release rigmarole like setting up versions, tests, tagging,
> committing, etc. I use it for simple snapshots.
>
> mvn install is the even simpler thing, as it deploys your artifacts to
> your local machine repo, where you can depend on them locally.
> Neither of those two update all the versions for you AFAIK. Sorry.
> It's usually for snapshots, where the version is already set to the
> right snapshot version by default.
>
> Getting to know the Maven plugins takes some work, but eventually you
> save more time and errors using them than doing it manually.
>
> distributionManagement is inherited from the standard Apache parent.
>
> On Tue, Jul 29, 2014 at 8:58 PM, Koert Kuipers <ko...@tresata.com> wrote:
> > all i want to do is 1) change the version number and 2) publish spark to
> our
> > internal maven repo.
> >
> > i just learned that the maven way what i want to do is a "deploy", not a
> > "release" (neither of which makes sense to me.. how about publish? but
> ok).
> > although me wanting to change a version number is apparently too much to
> > ask, without using a plugin (the maven-release-plugin?). thats fine. i
> will
> > ignore that and go change all the files by hand.
> >
> > so i now understand i need to do a mvn deploy, and there should be a
> > distributionManagement section somewhere in the pom. ok great i can try
> > adding that myself. but how come it isnt there by default? i mean, spark
> > does end up on mavem somehow?
> >
> >
> >
> >
> > On Mon, Jul 28, 2014 at 3:39 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >>
> >> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
> >>
> >>
> >> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
> >>>
> >>> This is not something you edit yourself. The Maven release plugin
> >>> manages setting all this. I think virtually everything you're worried
> >>> about is done for you by this plugin.
> >>>
> >>> Maven requires artifacts to set a version and it can't inherit one. I
> >>> feel like I understood the reason this is necessary at one point.
> >>>
> >>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >>> > and if i want to change the version, it seems i have to change it in
> >>> > all 23
> >>> > pom files? mhhh. is it mandatory for these sub-project pom files to
> >>> > repeat
> >>> > that version info? useful?
> >>> >
> >>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
> >>> > 23
> >>> >
> >>> >
> >>> >
> >>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
> >>> > wrote:
> >>> >>
> >>> >> hey we used to publish spark inhouse by simply overriding the
> >>> >> publishTo
> >>> >> setting. but now that we are integrated in SBT with maven i cannot
> >>> >> find it
> >>> >> anymore.
> >>> >>
> >>> >> i tried looking into the pom file, but after reading 1144 lines of
> xml
> >>> >> i
> >>> >> 1) havent found anything that looks like publishing
> >>> >> 2) i feel somewhat sick too
> >>> >> 3) i am considering alternative careers to developing...
> >>> >>
> >>> >> where am i supposed to look?
> >>> >> thanks for your help!
> >>> >
> >>> >
> >>
> >>
> >
>

Re: how to publish spark inhouse?

[Sean Owen <so...@cloudera.com>]

Regarding SBT: I don't think SBT has equivalents for some Maven
concepts like "runtime" scope. The resulting assembly-like JARs might
not be what's intended. I think that's part of why Maven is the build
of reference. You might be able to get the SBT build to do the right
thing with fiddling. It was my understanding that sbt-pom-reader
didn't quite support everything required to make the same artifact. So
I think SBT is not suitable for making release-like things, even for
an internal snapshot.

deploy is the simplest goal for publishing things like snapshots, yes.
It just makes artifacts and pushes them, without going through a lot
of release rigmarole like setting up versions, tests, tagging,
committing, etc. I use it for simple snapshots.

mvn install is the even simpler thing, as it deploys your artifacts to
your local machine repo, where you can depend on them locally.
Neither of those two update all the versions for you AFAIK. Sorry.
It's usually for snapshots, where the version is already set to the
right snapshot version by default.

Getting to know the Maven plugins takes some work, but eventually you
save more time and errors using them than doing it manually.

distributionManagement is inherited from the standard Apache parent.

On Tue, Jul 29, 2014 at 8:58 PM, Koert Kuipers <ko...@tresata.com> wrote:
> all i want to do is 1) change the version number and 2) publish spark to our
> internal maven repo.
>
> i just learned that the maven way what i want to do is a "deploy", not a
> "release" (neither of which makes sense to me.. how about publish? but ok).
> although me wanting to change a version number is apparently too much to
> ask, without using a plugin (the maven-release-plugin?). thats fine. i will
> ignore that and go change all the files by hand.
>
> so i now understand i need to do a mvn deploy, and there should be a
> distributionManagement section somewhere in the pom. ok great i can try
> adding that myself. but how come it isnt there by default? i mean, spark
> does end up on mavem somehow?
>
>
>
>
> On Mon, Jul 28, 2014 at 3:39 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>
>> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
>>
>>
>> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
>>>
>>> This is not something you edit yourself. The Maven release plugin
>>> manages setting all this. I think virtually everything you're worried
>>> about is done for you by this plugin.
>>>
>>> Maven requires artifacts to set a version and it can't inherit one. I
>>> feel like I understood the reason this is necessary at one point.
>>>
>>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>> > and if i want to change the version, it seems i have to change it in
>>> > all 23
>>> > pom files? mhhh. is it mandatory for these sub-project pom files to
>>> > repeat
>>> > that version info? useful?
>>> >
>>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
>>> > 23
>>> >
>>> >
>>> >
>>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
>>> > wrote:
>>> >>
>>> >> hey we used to publish spark inhouse by simply overriding the
>>> >> publishTo
>>> >> setting. but now that we are integrated in SBT with maven i cannot
>>> >> find it
>>> >> anymore.
>>> >>
>>> >> i tried looking into the pom file, but after reading 1144 lines of xml
>>> >> i
>>> >> 1) havent found anything that looks like publishing
>>> >> 2) i feel somewhat sick too
>>> >> 3) i am considering alternative careers to developing...
>>> >>
>>> >> where am i supposed to look?
>>> >> thanks for your help!
>>> >
>>> >
>>
>>
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

all i want to do is 1) change the version number and 2) publish spark to
our internal maven repo.

i just learned that the maven way what i want to do is a "deploy", not a
"release" (neither of which makes sense to me.. how about publish? but ok).
although me wanting to change a version number is apparently too much to
ask, without using a plugin (the maven-release-plugin?). thats fine. i will
ignore that and go change all the files by hand.

so i now understand i need to do a mvn deploy, and there should be a
distributionManagement section somewhere in the pom. ok great i can try
adding that myself. but how come it isnt there by default? i mean, spark
does end up on mavem somehow?




On Mon, Jul 28, 2014 at 3:39 PM, Koert Kuipers <ko...@tresata.com> wrote:

> ah ok thanks. guess i am gonna read up about maven-release-plugin then!
>
>
> On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
>
>> This is not something you edit yourself. The Maven release plugin
>> manages setting all this. I think virtually everything you're worried
>> about is done for you by this plugin.
>>
>> Maven requires artifacts to set a version and it can't inherit one. I
>> feel like I understood the reason this is necessary at one point.
>>
>> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com> wrote:
>> > and if i want to change the version, it seems i have to change it in
>> all 23
>> > pom files? mhhh. is it mandatory for these sub-project pom files to
>> repeat
>> > that version info? useful?
>> >
>> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
>> > 23
>> >
>> >
>> >
>> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
>> wrote:
>> >>
>> >> hey we used to publish spark inhouse by simply overriding the publishTo
>> >> setting. but now that we are integrated in SBT with maven i cannot
>> find it
>> >> anymore.
>> >>
>> >> i tried looking into the pom file, but after reading 1144 lines of xml
>> i
>> >> 1) havent found anything that looks like publishing
>> >> 2) i feel somewhat sick too
>> >> 3) i am considering alternative careers to developing...
>> >>
>> >> where am i supposed to look?
>> >> thanks for your help!
>> >
>> >
>>
>
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

ah ok thanks. guess i am gonna read up about maven-release-plugin then!


On Mon, Jul 28, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:

> This is not something you edit yourself. The Maven release plugin
> manages setting all this. I think virtually everything you're worried
> about is done for you by this plugin.
>
> Maven requires artifacts to set a version and it can't inherit one. I
> feel like I understood the reason this is necessary at one point.
>
> On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com> wrote:
> > and if i want to change the version, it seems i have to change it in all
> 23
> > pom files? mhhh. is it mandatory for these sub-project pom files to
> repeat
> > that version info? useful?
> >
> > spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
> > 23
> >
> >
> >
> > On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com>
> wrote:
> >>
> >> hey we used to publish spark inhouse by simply overriding the publishTo
> >> setting. but now that we are integrated in SBT with maven i cannot find
> it
> >> anymore.
> >>
> >> i tried looking into the pom file, but after reading 1144 lines of xml i
> >> 1) havent found anything that looks like publishing
> >> 2) i feel somewhat sick too
> >> 3) i am considering alternative careers to developing...
> >>
> >> where am i supposed to look?
> >> thanks for your help!
> >
> >
>

Re: how to publish spark inhouse?

[Sean Owen <so...@cloudera.com>]

This is not something you edit yourself. The Maven release plugin
manages setting all this. I think virtually everything you're worried
about is done for you by this plugin.

Maven requires artifacts to set a version and it can't inherit one. I
feel like I understood the reason this is necessary at one point.

On Mon, Jul 28, 2014 at 8:33 PM, Koert Kuipers <ko...@tresata.com> wrote:
> and if i want to change the version, it seems i have to change it in all 23
> pom files? mhhh. is it mandatory for these sub-project pom files to repeat
> that version info? useful?
>
> spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
> 23
>
>
>
> On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>
>> hey we used to publish spark inhouse by simply overriding the publishTo
>> setting. but now that we are integrated in SBT with maven i cannot find it
>> anymore.
>>
>> i tried looking into the pom file, but after reading 1144 lines of xml i
>> 1) havent found anything that looks like publishing
>> 2) i feel somewhat sick too
>> 3) i am considering alternative careers to developing...
>>
>> where am i supposed to look?
>> thanks for your help!
>
>

Re: how to publish spark inhouse?

[Koert Kuipers <ko...@tresata.com>]

and if i want to change the version, it seems i have to change it in all 23
pom files? mhhh. is it mandatory for these sub-project pom files to repeat
that version info? useful?

spark$ grep 1.1.0-SNAPSHOT * -r  | wc -l
23



On Mon, Jul 28, 2014 at 3:05 PM, Koert Kuipers <ko...@tresata.com> wrote:

> hey we used to publish spark inhouse by simply overriding the publishTo
> setting. but now that we are integrated in SBT with maven i cannot find it
> anymore.
>
> i tried looking into the pom file, but after reading 1144 lines of xml i
> 1) havent found anything that looks like publishing
> 2) i feel somewhat sick too
> 3) i am considering alternative careers to developing...
>
> where am i supposed to look?
> thanks for your help!
>