You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Kevin Burton <bu...@spinn3r.com> on 2014/11/08 19:47:13 UTC
Debian package for spark?
Are there debian packages for spark?
If not I plan on making one… I threw one together in about 20 minutes as
they are somewhat easy with maven and jdeb. But of course there are other
things I need to install like cassandra support and an init script.
So I figured I’d ask here first.
If not we will open source our packaging code and put it on github. It’s
about 50 lines of code :-P
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Debian package for spark?
Posted by varun sharma <va...@gmail.com>.
Hi Kevin,
Were you able to build spark with command "export MAVEN_OPTS="-Xmx2g
-XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m" && mvn -Pdeb
-DskipTests clean package" ?
I am getting the below error for all versions of spark(even 1.2.0):
Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on project
spark-assembly_2.10: Failed to create debian package
/opt/spark/spark-1.2.0/assembly/target/spark_1.2.0-${buildNumber}_all.deb:
Could not create deb package: Control file descriptor keys are invalid
[Version]. The following keys are mandatory [Package, Version, Section,
Priority, Architecture, Maintainer, Description]. Please check your
pom.xml/build.xml and your control file. -> [Help 1]
Thanks in advance.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Debian-package-for-spark-tp18410p20856.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Debian package for spark?
Posted by Mark Hamstra <ma...@clearstorydata.com>.
No change from 1.1.0 to 1.1.1-SNAPSHOT. The deb profile hasn't changed
since before the 1.0.2 release.
On Sat, Nov 8, 2014 at 3:12 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> Weird… I’m using a 1.1.0 source tar.gz …
>
> but if it’s fixed in 1.1.1 that’s good.
>
> On Sat, Nov 8, 2014 at 2:08 PM, Mark Hamstra <ma...@clearstorydata.com>
> wrote:
>
>> The building of the Debian package in Spark works just fine for me -- I
>> just did it using a clean check-out of 1.1.1-SNAPSHOT and `mvn -U -Pdeb
>> -DskipTests clean package`. There's likely something else amiss in your
>> build.
>>
>> Actually, that's not quite true. There is one small problem with the
>> Debian packaging that you should be aware of:
>>
>> https://issues.apache.org/jira/browse/SPARK-3624
>> https://github.com/apache/spark/pull/2477#issuecomment-58291272
>>
>> You should also know there is no such thing as standard Debian packages
>> or official debs for Spark, nor is it likely that there ever will be. What
>> is available was never intended as anything more than a convenient hack (or
>> a starting point for a custom hack) for Spark developers or users who need
>> a way to create a Spark package sufficient to use in a configuration
>> management system or something of that nature. A proper collection of debs
>> that divides Spark up into multiple parts, properly reflects inter-package
>> dependencies, relocates executables, configuration and libraries to conform
>> to the expectations of a larger system, etc. is something that the Apache
>> Spark Project does not do, probably won't do, and probably shouldn't do --
>> something like that is better handled by the distributors of OSes or larger
>> software systems like Apache Bigtop.
>>
>> On Sat, Nov 8, 2014 at 1:17 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>>> Another note for the official debs. ‘spark’ is a bad package name
>>> because of confusion with the spark programming lang based on ada.
>>>
>>> There are packages for this already named ‘spark’
>>>
>>> so I put mine as ‘apache-spark’
>>>
>>>
>>>
>>> On Sat, Nov 8, 2014 at 12:21 PM, Kevin Burton <bu...@spinn3r.com>
>>> wrote:
>>>
>>>> OK… here’s my version.
>>>>
>>>> https://github.com/spinn3r/spark-deb
>>>>
>>>> it’s just two files really. so if the standard spark packages get
>>>> fixed I’ll just switch to them.
>>>>
>>>> Doesn’t look like there’s an init script and the conf isn’t in /etc …
>>>>
>>>> On Sat, Nov 8, 2014 at 12:06 PM, Kevin Burton <bu...@spinn3r.com>
>>>> wrote:
>>>>
>>>>> looks like it doesn’t work:
>>>>>
>>>>> > [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
>>>>> project spark-assembly_2.10: Failed to create debian package
>>>>> /Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
>>>>> Could not create deb package: Control file descriptor keys are invalid
>>>>> [Version]. The following keys are mandatory [Package, Version, Section,
>>>>> Priority, Architecture, Maintainer, Description]. Please check your
>>>>> pom.xml/build.xml and your control file. -> [Help 1]
>>>>>
>>>>> On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com>
>>>>> wrote:
>>>>>
>>>>>> Nice! Not sure how I missed that. Building it now. If it has all
>>>>>> the init scripts and config in the right place I might use that.
>>>>>>
>>>>>> I might have to build a cassandra package too which adds cassandra
>>>>>> support.. I *think* at least.
>>>>>>
>>>>>> Maybe distribute this .deb with the standard downloads?
>>>>>>
>>>>>> Kevin
>>>>>>
>>>>>> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yep there is one have a look here
>>>>>>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>>>>>>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>>>>>>
>>>>>>> Are there debian packages for spark?
>>>>>>>>
>>>>>>>> If not I plan on making one… I threw one together in about 20
>>>>>>>> minutes as they are somewhat easy with maven and jdeb. But of course there
>>>>>>>> are other things I need to install like cassandra support and an init
>>>>>>>> script.
>>>>>>>>
>>>>>>>> So I figured I’d ask here first.
>>>>>>>>
>>>>>>>> If not we will open source our packaging code and put it on
>>>>>>>> github. It’s about 50 lines of code :-P
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Founder/CEO Spinn3r.com
>>>>>>>> Location: *San Francisco, CA*
>>>>>>>> blog: http://burtonator.wordpress.com
>>>>>>>> … or check out my Google+ profile
>>>>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>>>>> <http://spinn3r.com>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Founder/CEO Spinn3r.com
>>>>>> Location: *San Francisco, CA*
>>>>>> blog: http://burtonator.wordpress.com
>>>>>> … or check out my Google+ profile
>>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>>> <http://spinn3r.com>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Founder/CEO Spinn3r.com
>>>>> Location: *San Francisco, CA*
>>>>> blog: http://burtonator.wordpress.com
>>>>> … or check out my Google+ profile
>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>> <http://spinn3r.com>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Founder/CEO Spinn3r.com
>>>> Location: *San Francisco, CA*
>>>> blog: http://burtonator.wordpress.com
>>>> … or check out my Google+ profile
>>>> <https://plus.google.com/102718274791889610666/posts>
>>>> <http://spinn3r.com>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>> <http://spinn3r.com>
>>>
>>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
Re: Debian package for spark?
Posted by Kevin Burton <bu...@spinn3r.com>.
Weird… I’m using a 1.1.0 source tar.gz …
but if it’s fixed in 1.1.1 that’s good.
On Sat, Nov 8, 2014 at 2:08 PM, Mark Hamstra <ma...@clearstorydata.com>
wrote:
> The building of the Debian package in Spark works just fine for me -- I
> just did it using a clean check-out of 1.1.1-SNAPSHOT and `mvn -U -Pdeb
> -DskipTests clean package`. There's likely something else amiss in your
> build.
>
> Actually, that's not quite true. There is one small problem with the
> Debian packaging that you should be aware of:
>
> https://issues.apache.org/jira/browse/SPARK-3624
> https://github.com/apache/spark/pull/2477#issuecomment-58291272
>
> You should also know there is no such thing as standard Debian packages or
> official debs for Spark, nor is it likely that there ever will be. What is
> available was never intended as anything more than a convenient hack (or a
> starting point for a custom hack) for Spark developers or users who need a
> way to create a Spark package sufficient to use in a configuration
> management system or something of that nature. A proper collection of debs
> that divides Spark up into multiple parts, properly reflects inter-package
> dependencies, relocates executables, configuration and libraries to conform
> to the expectations of a larger system, etc. is something that the Apache
> Spark Project does not do, probably won't do, and probably shouldn't do --
> something like that is better handled by the distributors of OSes or larger
> software systems like Apache Bigtop.
>
> On Sat, Nov 8, 2014 at 1:17 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> Another note for the official debs. ‘spark’ is a bad package name
>> because of confusion with the spark programming lang based on ada.
>>
>> There are packages for this already named ‘spark’
>>
>> so I put mine as ‘apache-spark’
>>
>>
>>
>> On Sat, Nov 8, 2014 at 12:21 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>>> OK… here’s my version.
>>>
>>> https://github.com/spinn3r/spark-deb
>>>
>>> it’s just two files really. so if the standard spark packages get fixed
>>> I’ll just switch to them.
>>>
>>> Doesn’t look like there’s an init script and the conf isn’t in /etc …
>>>
>>> On Sat, Nov 8, 2014 at 12:06 PM, Kevin Burton <bu...@spinn3r.com>
>>> wrote:
>>>
>>>> looks like it doesn’t work:
>>>>
>>>> > [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
>>>> project spark-assembly_2.10: Failed to create debian package
>>>> /Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
>>>> Could not create deb package: Control file descriptor keys are invalid
>>>> [Version]. The following keys are mandatory [Package, Version, Section,
>>>> Priority, Architecture, Maintainer, Description]. Please check your
>>>> pom.xml/build.xml and your control file. -> [Help 1]
>>>>
>>>> On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com>
>>>> wrote:
>>>>
>>>>> Nice! Not sure how I missed that. Building it now. If it has all
>>>>> the init scripts and config in the right place I might use that.
>>>>>
>>>>> I might have to build a cassandra package too which adds cassandra
>>>>> support.. I *think* at least.
>>>>>
>>>>> Maybe distribute this .deb with the standard downloads?
>>>>>
>>>>> Kevin
>>>>>
>>>>> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Yep there is one have a look here
>>>>>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>>>>>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>>>>>
>>>>>> Are there debian packages for spark?
>>>>>>>
>>>>>>> If not I plan on making one… I threw one together in about 20
>>>>>>> minutes as they are somewhat easy with maven and jdeb. But of course there
>>>>>>> are other things I need to install like cassandra support and an init
>>>>>>> script.
>>>>>>>
>>>>>>> So I figured I’d ask here first.
>>>>>>>
>>>>>>> If not we will open source our packaging code and put it on github.
>>>>>>> It’s about 50 lines of code :-P
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Founder/CEO Spinn3r.com
>>>>>>> Location: *San Francisco, CA*
>>>>>>> blog: http://burtonator.wordpress.com
>>>>>>> … or check out my Google+ profile
>>>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>>>> <http://spinn3r.com>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Founder/CEO Spinn3r.com
>>>>> Location: *San Francisco, CA*
>>>>> blog: http://burtonator.wordpress.com
>>>>> … or check out my Google+ profile
>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>> <http://spinn3r.com>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Founder/CEO Spinn3r.com
>>>> Location: *San Francisco, CA*
>>>> blog: http://burtonator.wordpress.com
>>>> … or check out my Google+ profile
>>>> <https://plus.google.com/102718274791889610666/posts>
>>>> <http://spinn3r.com>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>> <http://spinn3r.com>
>>>
>>>
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Debian package for spark?
Posted by Mark Hamstra <ma...@clearstorydata.com>.
The building of the Debian package in Spark works just fine for me -- I
just did it using a clean check-out of 1.1.1-SNAPSHOT and `mvn -U -Pdeb
-DskipTests clean package`. There's likely something else amiss in your
build.
Actually, that's not quite true. There is one small problem with the
Debian packaging that you should be aware of:
https://issues.apache.org/jira/browse/SPARK-3624
https://github.com/apache/spark/pull/2477#issuecomment-58291272
You should also know there is no such thing as standard Debian packages or
official debs for Spark, nor is it likely that there ever will be. What is
available was never intended as anything more than a convenient hack (or a
starting point for a custom hack) for Spark developers or users who need a
way to create a Spark package sufficient to use in a configuration
management system or something of that nature. A proper collection of debs
that divides Spark up into multiple parts, properly reflects inter-package
dependencies, relocates executables, configuration and libraries to conform
to the expectations of a larger system, etc. is something that the Apache
Spark Project does not do, probably won't do, and probably shouldn't do --
something like that is better handled by the distributors of OSes or larger
software systems like Apache Bigtop.
On Sat, Nov 8, 2014 at 1:17 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> Another note for the official debs. ‘spark’ is a bad package name because
> of confusion with the spark programming lang based on ada.
>
> There are packages for this already named ‘spark’
>
> so I put mine as ‘apache-spark’
>
>
>
> On Sat, Nov 8, 2014 at 12:21 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> OK… here’s my version.
>>
>> https://github.com/spinn3r/spark-deb
>>
>> it’s just two files really. so if the standard spark packages get fixed
>> I’ll just switch to them.
>>
>> Doesn’t look like there’s an init script and the conf isn’t in /etc …
>>
>> On Sat, Nov 8, 2014 at 12:06 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>>> looks like it doesn’t work:
>>>
>>> > [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
>>> project spark-assembly_2.10: Failed to create debian package
>>> /Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
>>> Could not create deb package: Control file descriptor keys are invalid
>>> [Version]. The following keys are mandatory [Package, Version, Section,
>>> Priority, Architecture, Maintainer, Description]. Please check your
>>> pom.xml/build.xml and your control file. -> [Help 1]
>>>
>>> On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com>
>>> wrote:
>>>
>>>> Nice! Not sure how I missed that. Building it now. If it has all the
>>>> init scripts and config in the right place I might use that.
>>>>
>>>> I might have to build a cassandra package too which adds cassandra
>>>> support.. I *think* at least.
>>>>
>>>> Maybe distribute this .deb with the standard downloads?
>>>>
>>>> Kevin
>>>>
>>>> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
>>>> wrote:
>>>>
>>>>> Yep there is one have a look here
>>>>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>>>>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>>>>
>>>>> Are there debian packages for spark?
>>>>>>
>>>>>> If not I plan on making one… I threw one together in about 20 minutes
>>>>>> as they are somewhat easy with maven and jdeb. But of course there are
>>>>>> other things I need to install like cassandra support and an init script.
>>>>>>
>>>>>> So I figured I’d ask here first.
>>>>>>
>>>>>> If not we will open source our packaging code and put it on github.
>>>>>> It’s about 50 lines of code :-P
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Founder/CEO Spinn3r.com
>>>>>> Location: *San Francisco, CA*
>>>>>> blog: http://burtonator.wordpress.com
>>>>>> … or check out my Google+ profile
>>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>>> <http://spinn3r.com>
>>>>>>
>>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Founder/CEO Spinn3r.com
>>>> Location: *San Francisco, CA*
>>>> blog: http://burtonator.wordpress.com
>>>> … or check out my Google+ profile
>>>> <https://plus.google.com/102718274791889610666/posts>
>>>> <http://spinn3r.com>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>> <http://spinn3r.com>
>>>
>>>
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
Re: Debian package for spark?
Posted by Kevin Burton <bu...@spinn3r.com>.
Another note for the official debs. ‘spark’ is a bad package name because
of confusion with the spark programming lang based on ada.
There are packages for this already named ‘spark’
so I put mine as ‘apache-spark’
On Sat, Nov 8, 2014 at 12:21 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> OK… here’s my version.
>
> https://github.com/spinn3r/spark-deb
>
> it’s just two files really. so if the standard spark packages get fixed
> I’ll just switch to them.
>
> Doesn’t look like there’s an init script and the conf isn’t in /etc …
>
> On Sat, Nov 8, 2014 at 12:06 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> looks like it doesn’t work:
>>
>> > [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
>> project spark-assembly_2.10: Failed to create debian package
>> /Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
>> Could not create deb package: Control file descriptor keys are invalid
>> [Version]. The following keys are mandatory [Package, Version, Section,
>> Priority, Architecture, Maintainer, Description]. Please check your
>> pom.xml/build.xml and your control file. -> [Help 1]
>>
>> On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>>> Nice! Not sure how I missed that. Building it now. If it has all the
>>> init scripts and config in the right place I might use that.
>>>
>>> I might have to build a cassandra package too which adds cassandra
>>> support.. I *think* at least.
>>>
>>> Maybe distribute this .deb with the standard downloads?
>>>
>>> Kevin
>>>
>>> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
>>> wrote:
>>>
>>>> Yep there is one have a look here
>>>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>>>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>>>
>>>> Are there debian packages for spark?
>>>>>
>>>>> If not I plan on making one… I threw one together in about 20 minutes
>>>>> as they are somewhat easy with maven and jdeb. But of course there are
>>>>> other things I need to install like cassandra support and an init script.
>>>>>
>>>>> So I figured I’d ask here first.
>>>>>
>>>>> If not we will open source our packaging code and put it on github.
>>>>> It’s about 50 lines of code :-P
>>>>>
>>>>> --
>>>>>
>>>>> Founder/CEO Spinn3r.com
>>>>> Location: *San Francisco, CA*
>>>>> blog: http://burtonator.wordpress.com
>>>>> … or check out my Google+ profile
>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>> <http://spinn3r.com>
>>>>>
>>>>>
>>>
>>>
>>> --
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>> <http://spinn3r.com>
>>>
>>>
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Debian package for spark?
Posted by Kevin Burton <bu...@spinn3r.com>.
OK… here’s my version.
https://github.com/spinn3r/spark-deb
it’s just two files really. so if the standard spark packages get fixed
I’ll just switch to them.
Doesn’t look like there’s an init script and the conf isn’t in /etc …
On Sat, Nov 8, 2014 at 12:06 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> looks like it doesn’t work:
>
> > [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
> project spark-assembly_2.10: Failed to create debian package
> /Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
> Could not create deb package: Control file descriptor keys are invalid
> [Version]. The following keys are mandatory [Package, Version, Section,
> Priority, Architecture, Maintainer, Description]. Please check your
> pom.xml/build.xml and your control file. -> [Help 1]
>
> On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> Nice! Not sure how I missed that. Building it now. If it has all the
>> init scripts and config in the right place I might use that.
>>
>> I might have to build a cassandra package too which adds cassandra
>> support.. I *think* at least.
>>
>> Maybe distribute this .deb with the standard downloads?
>>
>> Kevin
>>
>> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
>> wrote:
>>
>>> Yep there is one have a look here
>>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>>
>>> Are there debian packages for spark?
>>>>
>>>> If not I plan on making one… I threw one together in about 20 minutes
>>>> as they are somewhat easy with maven and jdeb. But of course there are
>>>> other things I need to install like cassandra support and an init script.
>>>>
>>>> So I figured I’d ask here first.
>>>>
>>>> If not we will open source our packaging code and put it on github.
>>>> It’s about 50 lines of code :-P
>>>>
>>>> --
>>>>
>>>> Founder/CEO Spinn3r.com
>>>> Location: *San Francisco, CA*
>>>> blog: http://burtonator.wordpress.com
>>>> … or check out my Google+ profile
>>>> <https://plus.google.com/102718274791889610666/posts>
>>>> <http://spinn3r.com>
>>>>
>>>>
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Debian package for spark?
Posted by Kevin Burton <bu...@spinn3r.com>.
looks like it doesn’t work:
> [ERROR] Failed to execute goal org.vafer:jdeb:0.11:jdeb (default) on
project spark-assembly_2.10: Failed to create debian package
/Users/burton/Dropbox/projects-macbook-pro-2013-09/spark-1.1.0/assembly/target/spark_1.1.0-${buildNumber}_all.deb:
Could not create deb package: Control file descriptor keys are invalid
[Version]. The following keys are mandatory [Package, Version, Section,
Priority, Architecture, Maintainer, Description]. Please check your
pom.xml/build.xml and your control file. -> [Help 1]
On Sat, Nov 8, 2014 at 11:24 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> Nice! Not sure how I missed that. Building it now. If it has all the
> init scripts and config in the right place I might use that.
>
> I might have to build a cassandra package too which adds cassandra
> support.. I *think* at least.
>
> Maybe distribute this .deb with the standard downloads?
>
> Kevin
>
> On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com>
> wrote:
>
>> Yep there is one have a look here
>> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
>> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>>
>> Are there debian packages for spark?
>>>
>>> If not I plan on making one… I threw one together in about 20 minutes as
>>> they are somewhat easy with maven and jdeb. But of course there are other
>>> things I need to install like cassandra support and an init script.
>>>
>>> So I figured I’d ask here first.
>>>
>>> If not we will open source our packaging code and put it on github.
>>> It’s about 50 lines of code :-P
>>>
>>> --
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>> <http://spinn3r.com>
>>>
>>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Debian package for spark?
Posted by Kevin Burton <bu...@spinn3r.com>.
Nice! Not sure how I missed that. Building it now. If it has all the
init scripts and config in the right place I might use that.
I might have to build a cassandra package too which adds cassandra
support.. I *think* at least.
Maybe distribute this .deb with the standard downloads?
Kevin
On Sat, Nov 8, 2014 at 11:19 AM, Eugen Cepoi <ce...@gmail.com> wrote:
> Yep there is one have a look here
> http://spark.apache.org/docs/latest/building-with-maven.html#building-spark-debian-packages
> Le 8 nov. 2014 19:48, "Kevin Burton" <bu...@spinn3r.com> a écrit :
>
> Are there debian packages for spark?
>>
>> If not I plan on making one… I threw one together in about 20 minutes as
>> they are somewhat easy with maven and jdeb. But of course there are other
>> things I need to install like cassandra support and an init script.
>>
>> So I figured I’d ask here first.
>>
>> If not we will open source our packaging code and put it on github. It’s
>> about 50 lines of code :-P
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>