You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@amaterasu.apache.org by Shad Amez <sh...@gmail.com> on 2017/10/30 10:02:08 UTC
Unable to run the sample job
Hi All,
I started tinkering with source code of Amaterasu and just wanted to
confirm if I am missing any step.
Here are the steps that I followed :
1. Installed a single node mesos cluster (version 1.4) in Ubuntu 16.0.4
2. Generated the amaterasu tar file post building the project from source.
3. Tested if mesos by executing the following command :
mesos-execute --master=$MASTER --name="cluster-test" --command="sleep 5"
4. Ran the command for deploying the spark job using Amaterasu
./ama-start.sh --repo="https://github.com/shintoio/amaterasu-job-sample.git"
--branch="master" --env="test" --report="code"
Following is error log
repo: https://github.com/shintoio/amaterasu-job-sample.git
2017-10-30 14:00:09.761:INFO::main: Logging initialized @430ms
2017-10-30 14:00:09.829:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
2017-10-30 14:00:09.864:INFO:oejsh.ContextHandler:main: Started
o.e.j.s.ServletContextHandler@8c3b9d
{/,file:/home/shad/apps/apache-amaterasu-0.2.0-incubating/dist/,AVAILABLE}
2017-10-30 14:00:09.882:INFO:oejs.ServerConnector:main: Started
ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
2017-10-30 14:00:09.882:INFO:oejs.Server:main: Started @553ms
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
I1030 14:00:10.085204 16586 sched.cpp:232] Version: 1.4.0
I1030 14:00:10.088812 16630 sched.cpp:336] New master detected at
master@127.0.1.1:5050
I1030 14:00:10.089205 16630 sched.cpp:352] No credentials provided.
Attempting to register without authentication
I1030 14:00:10.090991 16624 sched.cpp:759] Framework registered with
e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
Exception in thread "Thread-13" while scanning for the next token
found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB)
for indentation)
in 'reader', line 2, column 1:
"name":"test",
^
at
org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:420)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)
at
org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.produce(ParserImpl.java:194)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:104)
at
org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:505)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:436)
at
org.apache.amaterasu.leader.utilities.DataLoader$.yamlToMap(DataLoader.scala:87)
at
org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$3.apply(DataLoader.scala:68)
at
org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$3.apply(DataLoader.scala:68)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
at
org.apache.amaterasu.leader.utilities.DataLoader$.getExecutorData(DataLoader.scala:68)
at
org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$$anonfun$resourceOffers$1.apply(JobScheduler.scala:163)
at
org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$$anonfun$resourceOffers$1.apply(JobScheduler.scala:128)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at
org.apache.amaterasu.leader.mesos.schedulers.JobScheduler.resourceOffers(JobScheduler.scala:128)
I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver
I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework
e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
Thanks,
Shad
Re: Unable to run the sample job
Posted by Yaniv Rodenski <ya...@shinto.io>.
Hi Shad,
While the plan is to have better support for running spark jars in future
versions, we currently have a workaround using the deps/jars.yml file.
You can see an example in the error-handling branch here:
https://github.com/shintoio/amaterasu-job-sample/tree/error-handling
The main thing to note is that the error.scala file is using an external
lib, which is being pulled from maven central (this is defined
in deps/jars.ym).
In order for this to work with spark programs, there are a couple of
caveats to note:
- Don't create a spark session/context in your jar, but pass in the one
defined in AmaContexct
- Exports will not work in this release
Other than those you should be OK.
Have a look at my slides from Dataworks Summit here:
https://www.slideshare.net/Hadoop_Summit/introduction-to-apache-amaterasu-incubating-cd-framework-for-your-big-data-pipelines,
and specifically the one on the planned 0.3.0 DSL to see where the current
thinking is at.
Cheers,
Yaniv
On Tue, Oct 31, 2017 at 9:36 PM, Shad Amez <sh...@gmail.com> wrote:
> Thanks Yaniv for the reply. The job now starts but completed with some
> errors.
>
> I wanted to know if the current implementation has the ability trigger the
> job from the Spark batch jar file, instead of reading from a github repo ?
> Or should this be considered as a feature request ?
>
> PFB the error logs.
>
> ===> Executor 0000000002-efce1eff-b351-4bc3-9bb2-776af04d3b3b registered
> ===> a provider for group spark was created
> ===> launching task: 0000000002
> ===> ================= started action start =================
> ===> val data = 1 to 1000
> ===> val rdd = sc.parallelize(data)
> ===> val odd = rdd.filter(n => n%2 != 0).toDF("number")
> ===> ================= finished action start =================
> ===> complete task: 0000000002
> ===> launching task: 0000000003
> ===> ================= started action start =================
> ===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number
> >
> 100")
> ===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
> outputRootPath}/dtatframe_res")
> ===> org.apache.spark.SparkException: Job aborted.
> ===> Executor 0000000003-ada561dd-f38b-4741-9ca5-eeb20780bbcf registered
> ===> a provider for group spark was created
> ===> launching task: 0000000003
> ===> ================= started action step2 =================
> ===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number
> >
> 100")
> ===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
> outputRootPath}/dtatframe_res")
> ===> org.apache.spark.SparkException: Job aborted.
> ===> Executor 0000000003-c65bcfd5-215b-46dc-9bd8-b38d31f50bb1 registered
> ===> a provider for group spark was created
> ===> launching task: 0000000003
> ===> ================= started action step2 =================
> ===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number
> >
> 100")
> ===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
> outputRootPath}/dtatframe_res")
> ===> org.apache.spark.SparkException: Job aborted.
> ===> moving to err action null
> 2017-10-31 10:53:37.730:INFO:oejs.ServerConnector:Thread-73: Stopped
> ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
> 2017-10-31 10:53:37.732:INFO:oejsh.ContextHandler:Thread-73: Stopped
> o.e.j.s.ServletContextHandler@8c3b9d{/,file:/home/shad/apps/
> apache-amaterasu-0.2.0-incubating/dist/,UNAVAILABLE}
> I1031 10:53:37.733896 36249 sched.cpp:2021] Asked to stop the driver
> I1031 10:53:37.734133 36249 sched.cpp:1203] Stopping framework
> afd772c2-509b-4782-a4c6-4cd9a2985cc2-0001
>
>
> W00t amaterasu job is finished!!!
>
> Thanks,
> Shad
>
> On Tue, Oct 31, 2017 at 5:02 AM, Yaniv Rodenski <ya...@shinto.io> wrote:
>
> > Hi Shad,
> >
> > sorry about that, there was indeed an issue with the job definition.
> > Should be fine now.
> >
> > Cheers,
> > Yaniv
> >
> > On Mon, Oct 30, 2017 at 9:02 PM, Shad Amez <sh...@gmail.com>
> wrote:
> >
> > > Hi All,
> > >
> > > I started tinkering with source code of Amaterasu and just wanted to
> > > confirm if I am missing any step.
> > >
> > > Here are the steps that I followed :
> > >
> > > 1. Installed a single node mesos cluster (version 1.4) in Ubuntu 16.0.4
> > > 2. Generated the amaterasu tar file post building the project from
> > source.
> > > 3. Tested if mesos by executing the following command :
> > > mesos-execute --master=$MASTER --name="cluster-test"
> --command="sleep
> > 5"
> > > 4. Ran the command for deploying the spark job using Amaterasu
> > > ./ama-start.sh --repo="https://github.com/
> shintoio/amaterasu-job-sample
> > .
> > > git"
> > > --branch="master" --env="test" --report="code"
> > >
> > > Following is error log
> > >
> > > repo: https://github.com/shintoio/amaterasu-job-sample.git
> > > 2017-10-30 14:00:09.761:INFO::main: Logging initialized @430ms
> > > 2017-10-30 14:00:09.829:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
> > > 2017-10-30 14:00:09.864:INFO:oejsh.ContextHandler:main: Started
> > > o.e.j.s.ServletContextHandler@8c3b9d
> > > {/,file:/home/shad/apps/apache-amaterasu-0.2.0-
> > incubating/dist/,AVAILABLE}
> > > 2017-10-30 14:00:09.882:INFO:oejs.ServerConnector:main: Started
> > > ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
> > > 2017-10-30 14:00:09.882:INFO:oejs.Server:main: Started @553ms
> > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> > further
> > > details.
> > > I1030 14:00:10.085204 16586 sched.cpp:232] Version: 1.4.0
> > > I1030 14:00:10.088812 16630 sched.cpp:336] New master detected at
> > > master@127.0.1.1:5050
> > > I1030 14:00:10.089205 16630 sched.cpp:352] No credentials provided.
> > > Attempting to register without authentication
> > > I1030 14:00:10.090991 16624 sched.cpp:759] Framework registered with
> > > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> > > Exception in thread "Thread-13" while scanning for the next token
> > > found character '\t(TAB)' that cannot start any token. (Do not use
> > \t(TAB)
> > > for indentation)
> > > in 'reader', line 2, column 1:
> > > "name":"test",
> > > ^
> > >
> > > at
> > > org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(
> > > ScannerImpl.java:420)
> > > at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(
> > ScannerImpl.java:226)
> > > at
> > > org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.
> > > produce(ParserImpl.java:194)
> > > at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
> > > at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(
> ParserImpl.java:147)
> > > at org.yaml.snakeyaml.composer.Composer.getSingleNode(
> Composer.java:104)
> > > at
> > > org.yaml.snakeyaml.constructor.BaseConstructor.
> > > getSingleData(BaseConstructor.java:122)
> > > at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:505)
> > > at org.yaml.snakeyaml.Yaml.load(Yaml.java:436)
> > > at
> > > org.apache.amaterasu.leader.utilities.DataLoader$.
> > > yamlToMap(DataLoader.scala:87)
> > > at
> > > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > > 3.apply(DataLoader.scala:68)
> > > at
> > > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > > 3.apply(DataLoader.scala:68)
> > > at
> > > scala.collection.TraversableLike$$anonfun$map$
> > > 1.apply(TraversableLike.scala:234)
> > > at
> > > scala.collection.TraversableLike$$anonfun$map$
> > > 1.apply(TraversableLike.scala:234)
> > > at
> > > scala.collection.IndexedSeqOptimized$class.
> foreach(IndexedSeqOptimized.
> > > scala:33)
> > > at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> > > at scala.collection.TraversableLike$class.map(
> TraversableLike.scala:234)
> > > at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
> > > at
> > > org.apache.amaterasu.leader.utilities.DataLoader$.
> > > getExecutorData(DataLoader.scala:68)
> > > at
> > > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > > $anonfun$resourceOffers$1.apply(JobScheduler.scala:163)
> > > at
> > > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > > $anonfun$resourceOffers$1.apply(JobScheduler.scala:128)
> > > at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> > > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> > > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > > at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> > > at
> > > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler.
> > > resourceOffers(JobScheduler.scala:128)
> > > I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver
> > > I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework
> > > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> > >
> > > Thanks,
> > > Shad
> > >
> >
> >
> >
> > --
> > Yaniv Rodenski
> >
>
--
Yaniv Rodenski
Re: Unable to run the sample job
Posted by Shad Amez <sh...@gmail.com>.
Thanks Yaniv for the reply. The job now starts but completed with some
errors.
I wanted to know if the current implementation has the ability trigger the
job from the Spark batch jar file, instead of reading from a github repo ?
Or should this be considered as a feature request ?
PFB the error logs.
===> Executor 0000000002-efce1eff-b351-4bc3-9bb2-776af04d3b3b registered
===> a provider for group spark was created
===> launching task: 0000000002
===> ================= started action start =================
===> val data = 1 to 1000
===> val rdd = sc.parallelize(data)
===> val odd = rdd.filter(n => n%2 != 0).toDF("number")
===> ================= finished action start =================
===> complete task: 0000000002
===> launching task: 0000000003
===> ================= started action start =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> Executor 0000000003-ada561dd-f38b-4741-9ca5-eeb20780bbcf registered
===> a provider for group spark was created
===> launching task: 0000000003
===> ================= started action step2 =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> Executor 0000000003-c65bcfd5-215b-46dc-9bd8-b38d31f50bb1 registered
===> a provider for group spark was created
===> launching task: 0000000003
===> ================= started action step2 =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===> highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.
outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> moving to err action null
2017-10-31 10:53:37.730:INFO:oejs.ServerConnector:Thread-73: Stopped
ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
2017-10-31 10:53:37.732:INFO:oejsh.ContextHandler:Thread-73: Stopped
o.e.j.s.ServletContextHandler@8c3b9d{/,file:/home/shad/apps/
apache-amaterasu-0.2.0-incubating/dist/,UNAVAILABLE}
I1031 10:53:37.733896 36249 sched.cpp:2021] Asked to stop the driver
I1031 10:53:37.734133 36249 sched.cpp:1203] Stopping framework
afd772c2-509b-4782-a4c6-4cd9a2985cc2-0001
W00t amaterasu job is finished!!!
Thanks,
Shad
On Tue, Oct 31, 2017 at 5:02 AM, Yaniv Rodenski <ya...@shinto.io> wrote:
> Hi Shad,
>
> sorry about that, there was indeed an issue with the job definition.
> Should be fine now.
>
> Cheers,
> Yaniv
>
> On Mon, Oct 30, 2017 at 9:02 PM, Shad Amez <sh...@gmail.com> wrote:
>
> > Hi All,
> >
> > I started tinkering with source code of Amaterasu and just wanted to
> > confirm if I am missing any step.
> >
> > Here are the steps that I followed :
> >
> > 1. Installed a single node mesos cluster (version 1.4) in Ubuntu 16.0.4
> > 2. Generated the amaterasu tar file post building the project from
> source.
> > 3. Tested if mesos by executing the following command :
> > mesos-execute --master=$MASTER --name="cluster-test" --command="sleep
> 5"
> > 4. Ran the command for deploying the spark job using Amaterasu
> > ./ama-start.sh --repo="https://github.com/shintoio/amaterasu-job-sample
> .
> > git"
> > --branch="master" --env="test" --report="code"
> >
> > Following is error log
> >
> > repo: https://github.com/shintoio/amaterasu-job-sample.git
> > 2017-10-30 14:00:09.761:INFO::main: Logging initialized @430ms
> > 2017-10-30 14:00:09.829:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
> > 2017-10-30 14:00:09.864:INFO:oejsh.ContextHandler:main: Started
> > o.e.j.s.ServletContextHandler@8c3b9d
> > {/,file:/home/shad/apps/apache-amaterasu-0.2.0-
> incubating/dist/,AVAILABLE}
> > 2017-10-30 14:00:09.882:INFO:oejs.ServerConnector:main: Started
> > ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
> > 2017-10-30 14:00:09.882:INFO:oejs.Server:main: Started @553ms
> > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> further
> > details.
> > I1030 14:00:10.085204 16586 sched.cpp:232] Version: 1.4.0
> > I1030 14:00:10.088812 16630 sched.cpp:336] New master detected at
> > master@127.0.1.1:5050
> > I1030 14:00:10.089205 16630 sched.cpp:352] No credentials provided.
> > Attempting to register without authentication
> > I1030 14:00:10.090991 16624 sched.cpp:759] Framework registered with
> > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> > Exception in thread "Thread-13" while scanning for the next token
> > found character '\t(TAB)' that cannot start any token. (Do not use
> \t(TAB)
> > for indentation)
> > in 'reader', line 2, column 1:
> > "name":"test",
> > ^
> >
> > at
> > org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(
> > ScannerImpl.java:420)
> > at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(
> ScannerImpl.java:226)
> > at
> > org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.
> > produce(ParserImpl.java:194)
> > at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
> > at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
> > at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:104)
> > at
> > org.yaml.snakeyaml.constructor.BaseConstructor.
> > getSingleData(BaseConstructor.java:122)
> > at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:505)
> > at org.yaml.snakeyaml.Yaml.load(Yaml.java:436)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$.
> > yamlToMap(DataLoader.scala:87)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > 3.apply(DataLoader.scala:68)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > 3.apply(DataLoader.scala:68)
> > at
> > scala.collection.TraversableLike$$anonfun$map$
> > 1.apply(TraversableLike.scala:234)
> > at
> > scala.collection.TraversableLike$$anonfun$map$
> > 1.apply(TraversableLike.scala:234)
> > at
> > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.
> > scala:33)
> > at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> > at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> > at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$.
> > getExecutorData(DataLoader.scala:68)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > $anonfun$resourceOffers$1.apply(JobScheduler.scala:163)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > $anonfun$resourceOffers$1.apply(JobScheduler.scala:128)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler.
> > resourceOffers(JobScheduler.scala:128)
> > I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver
> > I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework
> > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> >
> > Thanks,
> > Shad
> >
>
>
>
> --
> Yaniv Rodenski
>
Re: Unable to run the sample job
Posted by Shad Amez <sh...@gmail.com>.
Thanks Yaniv for the reply. The job now starts but completed with some
errors.
I wanted to know if the current implementation has the ability trigger the
job from the Spark batch jar file, instead of reading from a github repo ?
Or should this be considered as a feature request ?
PFB the error logs.
===> Executor 0000000002-efce1eff-b351-4bc3-9bb2-776af04d3b3b registered
===> a provider for group spark was created
===> launching task: 0000000002
===> ================= started action start =================
===> val data = 1 to 1000
===> val rdd = sc.parallelize(data)
===> val odd = rdd.filter(n => n%2 != 0).toDF("number")
===> ================= finished action start =================
===> complete task: 0000000002
===> launching task: 0000000003
===> ================= started action start =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===>
highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> Executor 0000000003-ada561dd-f38b-4741-9ca5-eeb20780bbcf registered
===> a provider for group spark was created
===> launching task: 0000000003
===> ================= started action step2 =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===>
highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> Executor 0000000003-c65bcfd5-215b-46dc-9bd8-b38d31f50bb1 registered
===> a provider for group spark was created
===> launching task: 0000000003
===> ================= started action step2 =================
===> val highNoDf = AmaContext.getDataFrame("start", "odd").where("number >
100")
===>
highNoDf.write.mode(SaveMode.Overwrite).json(s"${env.outputRootPath}/dtatframe_res")
===> org.apache.spark.SparkException: Job aborted.
===> moving to err action null
2017-10-31 10:53:37.730:INFO:oejs.ServerConnector:Thread-73: Stopped
ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
2017-10-31 10:53:37.732:INFO:oejsh.ContextHandler:Thread-73: Stopped
o.e.j.s.ServletContextHandler@8c3b9d
{/,file:/home/shad/apps/apache-amaterasu-0.2.0-incubating/dist/,UNAVAILABLE}
I1031 10:53:37.733896 36249 sched.cpp:2021] Asked to stop the driver
I1031 10:53:37.734133 36249 sched.cpp:1203] Stopping framework
afd772c2-509b-4782-a4c6-4cd9a2985cc2-0001
W00t amaterasu job is finished!!!
Thanks,
Shad
On Tue, Oct 31, 2017 at 5:02 AM, Yaniv Rodenski <ya...@shinto.io> wrote:
> Hi Shad,
>
> sorry about that, there was indeed an issue with the job definition.
> Should be fine now.
>
> Cheers,
> Yaniv
>
> On Mon, Oct 30, 2017 at 9:02 PM, Shad Amez <sh...@gmail.com> wrote:
>
> > Hi All,
> >
> > I started tinkering with source code of Amaterasu and just wanted to
> > confirm if I am missing any step.
> >
> > Here are the steps that I followed :
> >
> > 1. Installed a single node mesos cluster (version 1.4) in Ubuntu 16.0.4
> > 2. Generated the amaterasu tar file post building the project from
> source.
> > 3. Tested if mesos by executing the following command :
> > mesos-execute --master=$MASTER --name="cluster-test" --command="sleep
> 5"
> > 4. Ran the command for deploying the spark job using Amaterasu
> > ./ama-start.sh --repo="https://github.com/shintoio/amaterasu-job-sample
> .
> > git"
> > --branch="master" --env="test" --report="code"
> >
> > Following is error log
> >
> > repo: https://github.com/shintoio/amaterasu-job-sample.git
> > 2017-10-30 14:00:09.761:INFO::main: Logging initialized @430ms
> > 2017-10-30 14:00:09.829:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
> > 2017-10-30 14:00:09.864:INFO:oejsh.ContextHandler:main: Started
> > o.e.j.s.ServletContextHandler@8c3b9d
> > {/,file:/home/shad/apps/apache-amaterasu-0.2.0-
> incubating/dist/,AVAILABLE}
> > 2017-10-30 14:00:09.882:INFO:oejs.ServerConnector:main: Started
> > ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
> > 2017-10-30 14:00:09.882:INFO:oejs.Server:main: Started @553ms
> > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> further
> > details.
> > I1030 14:00:10.085204 16586 sched.cpp:232] Version: 1.4.0
> > I1030 14:00:10.088812 16630 sched.cpp:336] New master detected at
> > master@127.0.1.1:5050
> > I1030 14:00:10.089205 16630 sched.cpp:352] No credentials provided.
> > Attempting to register without authentication
> > I1030 14:00:10.090991 16624 sched.cpp:759] Framework registered with
> > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> > Exception in thread "Thread-13" while scanning for the next token
> > found character '\t(TAB)' that cannot start any token. (Do not use
> \t(TAB)
> > for indentation)
> > in 'reader', line 2, column 1:
> > "name":"test",
> > ^
> >
> > at
> > org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(
> > ScannerImpl.java:420)
> > at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(
> ScannerImpl.java:226)
> > at
> > org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.
> > produce(ParserImpl.java:194)
> > at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
> > at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
> > at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:104)
> > at
> > org.yaml.snakeyaml.constructor.BaseConstructor.
> > getSingleData(BaseConstructor.java:122)
> > at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:505)
> > at org.yaml.snakeyaml.Yaml.load(Yaml.java:436)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$.
> > yamlToMap(DataLoader.scala:87)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > 3.apply(DataLoader.scala:68)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> > 3.apply(DataLoader.scala:68)
> > at
> > scala.collection.TraversableLike$$anonfun$map$
> > 1.apply(TraversableLike.scala:234)
> > at
> > scala.collection.TraversableLike$$anonfun$map$
> > 1.apply(TraversableLike.scala:234)
> > at
> > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.
> > scala:33)
> > at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> > at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> > at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
> > at
> > org.apache.amaterasu.leader.utilities.DataLoader$.
> > getExecutorData(DataLoader.scala:68)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > $anonfun$resourceOffers$1.apply(JobScheduler.scala:163)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> > $anonfun$resourceOffers$1.apply(JobScheduler.scala:128)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> > at
> > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler.
> > resourceOffers(JobScheduler.scala:128)
> > I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver
> > I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework
> > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> >
> > Thanks,
> > Shad
> >
>
>
>
> --
> Yaniv Rodenski
>
Re: Unable to run the sample job
Posted by Yaniv Rodenski <ya...@shinto.io>.
Hi Shad,
sorry about that, there was indeed an issue with the job definition.
Should be fine now.
Cheers,
Yaniv
On Mon, Oct 30, 2017 at 9:02 PM, Shad Amez <sh...@gmail.com> wrote:
> Hi All,
>
> I started tinkering with source code of Amaterasu and just wanted to
> confirm if I am missing any step.
>
> Here are the steps that I followed :
>
> 1. Installed a single node mesos cluster (version 1.4) in Ubuntu 16.0.4
> 2. Generated the amaterasu tar file post building the project from source.
> 3. Tested if mesos by executing the following command :
> mesos-execute --master=$MASTER --name="cluster-test" --command="sleep 5"
> 4. Ran the command for deploying the spark job using Amaterasu
> ./ama-start.sh --repo="https://github.com/shintoio/amaterasu-job-sample.
> git"
> --branch="master" --env="test" --report="code"
>
> Following is error log
>
> repo: https://github.com/shintoio/amaterasu-job-sample.git
> 2017-10-30 14:00:09.761:INFO::main: Logging initialized @430ms
> 2017-10-30 14:00:09.829:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
> 2017-10-30 14:00:09.864:INFO:oejsh.ContextHandler:main: Started
> o.e.j.s.ServletContextHandler@8c3b9d
> {/,file:/home/shad/apps/apache-amaterasu-0.2.0-incubating/dist/,AVAILABLE}
> 2017-10-30 14:00:09.882:INFO:oejs.ServerConnector:main: Started
> ServerConnector@58ea606c{HTTP/1.1}{0.0.0.0:8000}
> 2017-10-30 14:00:09.882:INFO:oejs.Server:main: Started @553ms
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> I1030 14:00:10.085204 16586 sched.cpp:232] Version: 1.4.0
> I1030 14:00:10.088812 16630 sched.cpp:336] New master detected at
> master@127.0.1.1:5050
> I1030 14:00:10.089205 16630 sched.cpp:352] No credentials provided.
> Attempting to register without authentication
> I1030 14:00:10.090991 16624 sched.cpp:759] Framework registered with
> e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
> Exception in thread "Thread-13" while scanning for the next token
> found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB)
> for indentation)
> in 'reader', line 2, column 1:
> "name":"test",
> ^
>
> at
> org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(
> ScannerImpl.java:420)
> at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)
> at
> org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.
> produce(ParserImpl.java:194)
> at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
> at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
> at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:104)
> at
> org.yaml.snakeyaml.constructor.BaseConstructor.
> getSingleData(BaseConstructor.java:122)
> at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:505)
> at org.yaml.snakeyaml.Yaml.load(Yaml.java:436)
> at
> org.apache.amaterasu.leader.utilities.DataLoader$.
> yamlToMap(DataLoader.scala:87)
> at
> org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> 3.apply(DataLoader.scala:68)
> at
> org.apache.amaterasu.leader.utilities.DataLoader$$anonfun$
> 3.apply(DataLoader.scala:68)
> at
> scala.collection.TraversableLike$$anonfun$map$
> 1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$
> 1.apply(TraversableLike.scala:234)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.
> scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
> at
> org.apache.amaterasu.leader.utilities.DataLoader$.
> getExecutorData(DataLoader.scala:68)
> at
> org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> $anonfun$resourceOffers$1.apply(JobScheduler.scala:163)
> at
> org.apache.amaterasu.leader.mesos.schedulers.JobScheduler$
> $anonfun$resourceOffers$1.apply(JobScheduler.scala:128)
> at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> org.apache.amaterasu.leader.mesos.schedulers.JobScheduler.
> resourceOffers(JobScheduler.scala:128)
> I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver
> I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework
> e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002
>
> Thanks,
> Shad
>
--
Yaniv Rodenski