You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Chi Zewen <ch...@outlook.com> on 2017/10/26 03:06:34 UTC
Build Failure when exec "gradle spark-rpm"
Hi,
Nice to meet you all. I’m Zewen, a trainee working in China.
I’m trying to package spark into rpm packages by using “./gradlew” & “gradle spark-rpm” on docker(centos7).
I tried many times and failed with the following output:
[INFO] Compiling 26 source files to /root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/spark-2.1.0/yarn/target/scala-2.11/classes at 1508926971435
[WARNING] warning: Class org.mortbay.jetty.Server not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.Connector not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.handler.ContextHandlerCollection not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not found - continuing with a stub.
[WARNING] warning: Class org.mortbay.jetty.Connector not found - continuing with a stub.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Skipping Spark Project External Flume
[INFO] This project has been banned from the build due to previous failures.
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Skipping Spark Integration for Kafka 0.8
[INFO] This project has been banned from the build due to previous failures.
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Skipping Spark Integration for Kafka 0.10
[INFO] This project has been banned from the build due to previous failures.
[INFO] ------------------------------------------------------------------------
[INFO] Spark Project Parent POM ........................... SUCCESS [ 5.967 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 9.337 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 24.555 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 26.167 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 18.151 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 32.972 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 20.188 s]
[INFO] Spark Project Core ................................. SUCCESS [07:24 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [01:10 min]
[INFO] Spark Project GraphX ............................... SUCCESS [01:59 min]
[INFO] Spark Project Streaming ............................ SUCCESS [04:01 min]
[INFO] Spark Project Catalyst ............................. FAILURE [03:37 min]
[INFO] Spark Project SQL .................................. SKIPPED
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SUCCESS [ 14.261 s]
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 15.728 s]
[INFO] Spark Project YARN ................................. SKIPPED
[INFO] Spark Project Hive Thrift Server ................... SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Flume Sink .................. SUCCESS [ 36.498 s]
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
[INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:09 min (Wall Clock)
[INFO] Finished at: 2017-10-25T10:24:20+00:00
[INFO] Final Memory: 64M/683M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-catalyst_2.11: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 137 (Exit value: 137) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-catalyst_2.11
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
:spark-rpm FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462
* What went wrong:
Execution failed for task ':spark-rpm'.
> Process 'command 'rpmbuild'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Thanks,
Zewen Chi
Re: Build Failure when exec "gradle spark-rpm"
Posted by Chi Zewen <ch...@outlook.com>.
Hi Evans,
Thanks for help! After changing RAM limit of docker into 6GB, I got a 'BUILD SUCCESS'. Before this the RAM limit of my docker was set to 2GB :|
With regards,
Zewen Chi
在 2017年10月27日，下午2:28，Evans Ye <ev...@apache.org>> 写道：
I was able to build spark-rpm with exactly same command you provided.
Maybe you can check the sha256 of docker image. Mine is 059ac23e10e7.
2017-10-27 14:14 GMT+08:00 Konstantin Boudnik <co...@apache.org>>:
Ah great! I was about to start digging into our CI settings ;)
What wass the limit of RAM that fixed the issue, BTW?
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Thu, Oct 26, 2017 at 10:13 PM, Chi Zewen <ch...@outlook.com>> wrote:
> Hi Cos,
>
> I’ve solved this problem by modifying the memory limit of docker. I really don’t know what I would have done without your help. Thanks a lot. I really appreciate it.
>
> Regards,
> Zewen
>
>
>> 在 2017年10月26日，下午12:28，Konstantin Boudnik <co...@apache.org>> 写道：
>>
>> Are you using our recommended toolchain? Or the readily available docker images we use for the CI builds?
>> --
>> Regards,
>> Cos
>>
>> On October 25, 2017 8:06:34 PM PDT, Chi Zewen <ch...@outlook.com>> wrote:
>>> Hi,
>>>
>>> Nice to meet you all. I’m Zewen, a trainee working in China.
>>>
>>> I’m trying to package spark into rpm packages by using “./gradlew” &
>>> “gradle spark-rpm” on docker(centos7).
>>> I tried many times and failed with the following output:
>>>
>>> [INFO] Compiling 26 source files to
>>> /root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/spark-2.1.0/yarn/target/scala-2.11/classes
>>> at 1508926971435
>>> [WARNING] warning: Class org.mortbay.jetty.Server not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>>> found - continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class
>>> org.mortbay.jetty.handler.ContextHandlerCollection not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>>> found - continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
>>> continuing with a stub.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Reactor Summary:
>>> [INFO]
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Project External Flume
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Integration for Kafka 0.8
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Integration for Kafka 0.10
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Spark Project Parent POM ........................... SUCCESS [
>>> 5.967 s]
>>> [INFO] Spark Project Tags ................................. SUCCESS [
>>> 9.337 s]
>>> [INFO] Spark Project Sketch ............................... SUCCESS [
>>> 24.555 s]
>>> [INFO] Spark Project Networking ........................... SUCCESS [
>>> 26.167 s]
>>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
>>> 18.151 s]
>>> [INFO] Spark Project Unsafe ............................... SUCCESS [
>>> 32.972 s]
>>> [INFO] Spark Project Launcher ............................. SUCCESS [
>>> 20.188 s]
>>> [INFO] Spark Project Core ................................. SUCCESS
>>> [07:24 min]
>>> [INFO] Spark Project ML Local Library ..................... SUCCESS
>>> [01:10 min]
>>> [INFO] Spark Project GraphX ............................... SUCCESS
>>> [01:59 min]
>>> [INFO] Spark Project Streaming ............................ SUCCESS
>>> [04:01 min]
>>> [INFO] Spark Project Catalyst ............................. FAILURE
>>> [03:37 min]
>>> [INFO] Spark Project SQL .................................. SKIPPED
>>> [INFO] Spark Project ML Library ........................... SKIPPED
>>> [INFO] Spark Project Tools ................................ SUCCESS [
>>> 14.261 s]
>>> [INFO] Spark Project Hive ................................. SKIPPED
>>> [INFO] Spark Project REPL ................................. SKIPPED
>>> [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [
>>> 15.728 s]
>>> [INFO] Spark Project YARN ................................. SKIPPED
>>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
>>> [INFO] Spark Project Assembly ............................. SKIPPED
>>> [INFO] Spark Project External Flume Sink .................. SUCCESS [
>>> 36.498 s]
>>> [INFO] Spark Project External Flume ....................... SKIPPED
>>> [INFO] Spark Project External Flume Assembly .............. SKIPPED
>>> [INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
>>> [INFO] Spark Project Examples ............................. SKIPPED
>>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
>>> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
>>> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
>>> [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 15:09 min (Wall Clock)
>>> [INFO] Finished at: 2017-10-25T10:24:20+00:00
>>> [INFO] Final Memory: 64M/683M
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] Failed to execute goal
>>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile
>>> (scala-compile-first) on project spark-catalyst_2.11: wrap:
>>> org.apache.commons.exec.ExecuteException: Process exited with an error:
>>> 137 (Exit value: 137) -> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>> the -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>>> [ERROR]
>>> [ERROR] After correcting the problems, you can resume the build with
>>> the command
>>> [ERROR] mvn <goals> -rf :spark-catalyst_2.11
>>>
>>>
>>> RPM build errors:
>>> error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
>>> Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
>>> :spark-rpm FAILED
>>>
>>> FAILURE: Build failed with an exception.
>>>
>>> * Where:
>>> Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462
>>>
>>> * What went wrong:
>>> Execution failed for task ':spark-rpm'.
>>>> Process 'command 'rpmbuild'' finished with non-zero exit value 1
>>>
>>> * Try:
>>> Run with --stacktrace option to get the stack trace. Run with --info or
>>> --debug option to get more log output.
>>>
>>> BUILD FAILED
>>>
>>> Thanks,
>>>
>>> Zewen Chi
>
Re: Build Failure when exec "gradle spark-rpm"
Posted by Evans Ye <ev...@apache.org>.
I was able to build spark-rpm with exactly same command you provided.
Maybe you can check the sha256 of docker image. Mine is 059ac23e10e7.
2017-10-27 14:14 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> Ah great! I was about to start digging into our CI settings ;)
> What wass the limit of RAM that fixed the issue, BTW?
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>
> On Thu, Oct 26, 2017 at 10:13 PM, Chi Zewen <ch...@outlook.com> wrote:
> > Hi Cos,
> >
> > I’ve solved this problem by modifying the memory limit of docker. I
> really don’t know what I would have done without your help. Thanks a lot. I
> really appreciate it.
> >
> > Regards,
> > Zewen
> >
> >
> >> 在 2017年10月26日，下午12:28，Konstantin Boudnik <co...@apache.org> 写道：
> >>
> >> Are you using our recommended toolchain? Or the readily available
> docker images we use for the CI builds?
> >> --
> >> Regards,
> >> Cos
> >>
> >> On October 25, 2017 8:06:34 PM PDT, Chi Zewen <ch...@outlook.com>
> wrote:
> >>> Hi,
> >>>
> >>> Nice to meet you all. I’m Zewen, a trainee working in China.
> >>>
> >>> I’m trying to package spark into rpm packages by using “./gradlew” &
> >>> “gradle spark-rpm” on docker(centos7).
> >>> I tried many times and failed with the following output:
> >>>
> >>> [INFO] Compiling 26 source files to
> >>> /root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/
> spark-2.1.0/yarn/target/scala-2.11/classes
> >>> at 1508926971435
> >>> [WARNING] warning: Class org.mortbay.jetty.Server not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
> >>> found - continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class
> >>> org.mortbay.jetty.handler.ContextHandlerCollection not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
> >>> continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
> >>> found - continuing with a stub.
> >>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
> >>> continuing with a stub.
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Reactor Summary:
> >>> [INFO]
> >>> [INFO]
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Skipping Spark Project External Flume
> >>> [INFO] This project has been banned from the build due to previous
> >>> failures.
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO]
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Skipping Spark Integration for Kafka 0.8
> >>> [INFO] This project has been banned from the build due to previous
> >>> failures.
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO]
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Skipping Spark Integration for Kafka 0.10
> >>> [INFO] This project has been banned from the build due to previous
> >>> failures.
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Spark Project Parent POM ........................... SUCCESS [
> >>> 5.967 s]
> >>> [INFO] Spark Project Tags ................................. SUCCESS [
> >>> 9.337 s]
> >>> [INFO] Spark Project Sketch ............................... SUCCESS [
> >>> 24.555 s]
> >>> [INFO] Spark Project Networking ........................... SUCCESS [
> >>> 26.167 s]
> >>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
> >>> 18.151 s]
> >>> [INFO] Spark Project Unsafe ............................... SUCCESS [
> >>> 32.972 s]
> >>> [INFO] Spark Project Launcher ............................. SUCCESS [
> >>> 20.188 s]
> >>> [INFO] Spark Project Core ................................. SUCCESS
> >>> [07:24 min]
> >>> [INFO] Spark Project ML Local Library ..................... SUCCESS
> >>> [01:10 min]
> >>> [INFO] Spark Project GraphX ............................... SUCCESS
> >>> [01:59 min]
> >>> [INFO] Spark Project Streaming ............................ SUCCESS
> >>> [04:01 min]
> >>> [INFO] Spark Project Catalyst ............................. FAILURE
> >>> [03:37 min]
> >>> [INFO] Spark Project SQL .................................. SKIPPED
> >>> [INFO] Spark Project ML Library ........................... SKIPPED
> >>> [INFO] Spark Project Tools ................................ SUCCESS [
> >>> 14.261 s]
> >>> [INFO] Spark Project Hive ................................. SKIPPED
> >>> [INFO] Spark Project REPL ................................. SKIPPED
> >>> [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [
> >>> 15.728 s]
> >>> [INFO] Spark Project YARN ................................. SKIPPED
> >>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
> >>> [INFO] Spark Project Assembly ............................. SKIPPED
> >>> [INFO] Spark Project External Flume Sink .................. SUCCESS [
> >>> 36.498 s]
> >>> [INFO] Spark Project External Flume ....................... SKIPPED
> >>> [INFO] Spark Project External Flume Assembly .............. SKIPPED
> >>> [INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
> >>> [INFO] Spark Project Examples ............................. SKIPPED
> >>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
> >>> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
> >>> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
> >>> [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] BUILD FAILURE
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [INFO] Total time: 15:09 min (Wall Clock)
> >>> [INFO] Finished at: 2017-10-25T10:24:20+00:00
> >>> [INFO] Final Memory: 64M/683M
> >>> [INFO]
> >>> ------------------------------------------------------------
> ------------
> >>> [ERROR] Failed to execute goal
> >>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> >>> (scala-compile-first) on project spark-catalyst_2.11: wrap:
> >>> org.apache.commons.exec.ExecuteException: Process exited with an
> error:
> >>> 137 (Exit value: 137) -> [Help 1]
> >>> [ERROR]
> >>> [ERROR] To see the full stack trace of the errors, re-run Maven with
> >>> the -e switch.
> >>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >>> [ERROR]
> >>> [ERROR] For more information about the errors and possible solutions,
> >>> please read the following articles:
> >>> [ERROR] [Help 1]
> >>> http://cwiki.apache.org/confluence/display/MAVEN/
> MojoExecutionException
> >>> [ERROR]
> >>> [ERROR] After correcting the problems, you can resume the build with
> >>> the command
> >>> [ERROR] mvn <goals> -rf :spark-catalyst_2.11
> >>>
> >>>
> >>> RPM build errors:
> >>> error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
> >>> Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
> >>> :spark-rpm FAILED
> >>>
> >>> FAILURE: Build failed with an exception.
> >>>
> >>> * Where:
> >>> Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462
> >>>
> >>> * What went wrong:
> >>> Execution failed for task ':spark-rpm'.
> >>>> Process 'command 'rpmbuild'' finished with non-zero exit value 1
> >>>
> >>> * Try:
> >>> Run with --stacktrace option to get the stack trace. Run with --info or
> >>> --debug option to get more log output.
> >>>
> >>> BUILD FAILED
> >>>
> >>> Thanks,
> >>>
> >>> Zewen Chi
> >
>
Re: Build Failure when exec "gradle spark-rpm"
Posted by Konstantin Boudnik <co...@apache.org>.
Ah great! I was about to start digging into our CI settings ;)
What wass the limit of RAM that fixed the issue, BTW?
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Thu, Oct 26, 2017 at 10:13 PM, Chi Zewen <ch...@outlook.com> wrote:
> Hi Cos,
>
> I’ve solved this problem by modifying the memory limit of docker. I really don’t know what I would have done without your help. Thanks a lot. I really appreciate it.
>
> Regards,
> Zewen
>
>
>> 在 2017年10月26日，下午12:28，Konstantin Boudnik <co...@apache.org> 写道：
>>
>> Are you using our recommended toolchain? Or the readily available docker images we use for the CI builds?
>> --
>> Regards,
>> Cos
>>
>> On October 25, 2017 8:06:34 PM PDT, Chi Zewen <ch...@outlook.com> wrote:
>>> Hi,
>>>
>>> Nice to meet you all. I’m Zewen, a trainee working in China.
>>>
>>> I’m trying to package spark into rpm packages by using “./gradlew” &
>>> “gradle spark-rpm” on docker(centos7).
>>> I tried many times and failed with the following output:
>>>
>>> [INFO] Compiling 26 source files to
>>> /root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/spark-2.1.0/yarn/target/scala-2.11/classes
>>> at 1508926971435
>>> [WARNING] warning: Class org.mortbay.jetty.Server not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>>> found - continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class
>>> org.mortbay.jetty.handler.ContextHandlerCollection not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>>> continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>>> found - continuing with a stub.
>>> [WARNING] warning: Class org.mortbay.jetty.Connector not found -
>>> continuing with a stub.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Reactor Summary:
>>> [INFO]
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Project External Flume
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Integration for Kafka 0.8
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Skipping Spark Integration for Kafka 0.10
>>> [INFO] This project has been banned from the build due to previous
>>> failures.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Spark Project Parent POM ........................... SUCCESS [
>>> 5.967 s]
>>> [INFO] Spark Project Tags ................................. SUCCESS [
>>> 9.337 s]
>>> [INFO] Spark Project Sketch ............................... SUCCESS [
>>> 24.555 s]
>>> [INFO] Spark Project Networking ........................... SUCCESS [
>>> 26.167 s]
>>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
>>> 18.151 s]
>>> [INFO] Spark Project Unsafe ............................... SUCCESS [
>>> 32.972 s]
>>> [INFO] Spark Project Launcher ............................. SUCCESS [
>>> 20.188 s]
>>> [INFO] Spark Project Core ................................. SUCCESS
>>> [07:24 min]
>>> [INFO] Spark Project ML Local Library ..................... SUCCESS
>>> [01:10 min]
>>> [INFO] Spark Project GraphX ............................... SUCCESS
>>> [01:59 min]
>>> [INFO] Spark Project Streaming ............................ SUCCESS
>>> [04:01 min]
>>> [INFO] Spark Project Catalyst ............................. FAILURE
>>> [03:37 min]
>>> [INFO] Spark Project SQL .................................. SKIPPED
>>> [INFO] Spark Project ML Library ........................... SKIPPED
>>> [INFO] Spark Project Tools ................................ SUCCESS [
>>> 14.261 s]
>>> [INFO] Spark Project Hive ................................. SKIPPED
>>> [INFO] Spark Project REPL ................................. SKIPPED
>>> [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [
>>> 15.728 s]
>>> [INFO] Spark Project YARN ................................. SKIPPED
>>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
>>> [INFO] Spark Project Assembly ............................. SKIPPED
>>> [INFO] Spark Project External Flume Sink .................. SUCCESS [
>>> 36.498 s]
>>> [INFO] Spark Project External Flume ....................... SKIPPED
>>> [INFO] Spark Project External Flume Assembly .............. SKIPPED
>>> [INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
>>> [INFO] Spark Project Examples ............................. SKIPPED
>>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
>>> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
>>> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
>>> [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 15:09 min (Wall Clock)
>>> [INFO] Finished at: 2017-10-25T10:24:20+00:00
>>> [INFO] Final Memory: 64M/683M
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] Failed to execute goal
>>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile
>>> (scala-compile-first) on project spark-catalyst_2.11: wrap:
>>> org.apache.commons.exec.ExecuteException: Process exited with an error:
>>> 137 (Exit value: 137) -> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>> the -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>>> [ERROR]
>>> [ERROR] After correcting the problems, you can resume the build with
>>> the command
>>> [ERROR] mvn <goals> -rf :spark-catalyst_2.11
>>>
>>>
>>> RPM build errors:
>>> error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
>>> Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
>>> :spark-rpm FAILED
>>>
>>> FAILURE: Build failed with an exception.
>>>
>>> * Where:
>>> Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462
>>>
>>> * What went wrong:
>>> Execution failed for task ':spark-rpm'.
>>>> Process 'command 'rpmbuild'' finished with non-zero exit value 1
>>>
>>> * Try:
>>> Run with --stacktrace option to get the stack trace. Run with --info or
>>> --debug option to get more log output.
>>>
>>> BUILD FAILED
>>>
>>> Thanks,
>>>
>>> Zewen Chi
>
Re: Build Failure when exec "gradle spark-rpm"
Posted by Konstantin Boudnik <co...@apache.org>.
Are you using our recommended toolchain? Or the readily available docker images we use for the CI builds?
--
Regards,
Cos
On October 25, 2017 8:06:34 PM PDT, Chi Zewen <ch...@outlook.com> wrote:
>Hi,
>
>Nice to meet you all. I’m Zewen, a trainee working in China.
>
>I’m trying to package spark into rpm packages by using “./gradlew” &
>“gradle spark-rpm” on docker(centos7).
>I tried many times and failed with the following output:
>
>[INFO] Compiling 26 source files to
>/root/host/runtime/bigtop-rel-1.2.0/build/spark/rpm/BUILD/spark-2.1.0/yarn/target/scala-2.11/classes
>at 1508926971435
>[WARNING] warning: Class org.mortbay.jetty.Server not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>found - continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.Connector not found -
>continuing with a stub.
>[WARNING] warning: Class
>org.mortbay.jetty.handler.ContextHandlerCollection not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.servlet.Context not found -
>continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.webapp.WebAppContext not
>found - continuing with a stub.
>[WARNING] warning: Class org.mortbay.jetty.Connector not found -
>continuing with a stub.
>[INFO]
>------------------------------------------------------------------------
>[INFO] Reactor Summary:
>[INFO]
>[INFO]
>[INFO]
>------------------------------------------------------------------------
>[INFO] Skipping Spark Project External Flume
>[INFO] This project has been banned from the build due to previous
>failures.
>[INFO]
>------------------------------------------------------------------------
>[INFO]
>[INFO]
>------------------------------------------------------------------------
>[INFO] Skipping Spark Integration for Kafka 0.8
>[INFO] This project has been banned from the build due to previous
>failures.
>[INFO]
>------------------------------------------------------------------------
>[INFO]
>[INFO]
>------------------------------------------------------------------------
>[INFO] Skipping Spark Integration for Kafka 0.10
>[INFO] This project has been banned from the build due to previous
>failures.
>[INFO]
>------------------------------------------------------------------------
>[INFO] Spark Project Parent POM ........................... SUCCESS [
>5.967 s]
>[INFO] Spark Project Tags ................................. SUCCESS [
>9.337 s]
>[INFO] Spark Project Sketch ............................... SUCCESS [
>24.555 s]
>[INFO] Spark Project Networking ........................... SUCCESS [
>26.167 s]
>[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
>18.151 s]
>[INFO] Spark Project Unsafe ............................... SUCCESS [
>32.972 s]
>[INFO] Spark Project Launcher ............................. SUCCESS [
>20.188 s]
>[INFO] Spark Project Core ................................. SUCCESS
>[07:24 min]
>[INFO] Spark Project ML Local Library ..................... SUCCESS
>[01:10 min]
>[INFO] Spark Project GraphX ............................... SUCCESS
>[01:59 min]
>[INFO] Spark Project Streaming ............................ SUCCESS
>[04:01 min]
>[INFO] Spark Project Catalyst ............................. FAILURE
>[03:37 min]
>[INFO] Spark Project SQL .................................. SKIPPED
>[INFO] Spark Project ML Library ........................... SKIPPED
>[INFO] Spark Project Tools ................................ SUCCESS [
>14.261 s]
>[INFO] Spark Project Hive ................................. SKIPPED
>[INFO] Spark Project REPL ................................. SKIPPED
>[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [
>15.728 s]
>[INFO] Spark Project YARN ................................. SKIPPED
>[INFO] Spark Project Hive Thrift Server ................... SKIPPED
>[INFO] Spark Project Assembly ............................. SKIPPED
>[INFO] Spark Project External Flume Sink .................. SUCCESS [
>36.498 s]
>[INFO] Spark Project External Flume ....................... SKIPPED
>[INFO] Spark Project External Flume Assembly .............. SKIPPED
>[INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
>[INFO] Spark Project Examples ............................. SKIPPED
>[INFO] Spark Project External Kafka Assembly .............. SKIPPED
>[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
>[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
>[INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
>[INFO]
>------------------------------------------------------------------------
>[INFO] BUILD FAILURE
>[INFO]
>------------------------------------------------------------------------
>[INFO] Total time: 15:09 min (Wall Clock)
>[INFO] Finished at: 2017-10-25T10:24:20+00:00
>[INFO] Final Memory: 64M/683M
>[INFO]
>------------------------------------------------------------------------
>[ERROR] Failed to execute goal
>net.alchim31.maven:scala-maven-plugin:3.2.2:compile
>(scala-compile-first) on project spark-catalyst_2.11: wrap:
>org.apache.commons.exec.ExecuteException: Process exited with an error:
>137 (Exit value: 137) -> [Help 1]
>[ERROR]
>[ERROR] To see the full stack trace of the errors, re-run Maven with
>the -e switch.
>[ERROR] Re-run Maven using the -X switch to enable full debug logging.
>[ERROR]
>[ERROR] For more information about the errors and possible solutions,
>please read the following articles:
>[ERROR] [Help 1]
>http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>[ERROR]
>[ERROR] After correcting the problems, you can resume the build with
>the command
>[ERROR] mvn <goals> -rf :spark-catalyst_2.11
>
>
>RPM build errors:
>error: Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
> Bad exit status from /var/tmp/rpm-tmp.mjc1Oe (%build)
>:spark-rpm FAILED
>
>FAILURE: Build failed with an exception.
>
>* Where:
>Script '/root/host/runtime/bigtop-rel-1.2.0/packages.gradle' line: 462
>
>* What went wrong:
>Execution failed for task ':spark-rpm'.
>> Process 'command 'rpmbuild'' finished with non-zero exit value 1
>
>* Try:
>Run with --stacktrace option to get the stack trace. Run with --info or
>--debug option to get more log output.
>
>BUILD FAILED
>
>Thanks,
>
>Zewen Chi