You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@myriad.apache.org by Bill Sparks <js...@cray.com> on 2015/08/19 16:08:25 UTC
myriad scheduler startup with HDP2.7
I'm sure this is been resolved, but I've been triaging why I'm getting the following error on resourcemanager startup. Everything on the configuration side looks correct, but I must have missed something.
2015-08-19 08:53:04,718 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.NoSuchMethodError: com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/InputStream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:299)
at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:14)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2011)
at com.ebay.myriad.Main.initialize(Main.java:70)
at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32)
at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76)
at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:572)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:972)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:259)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1202)
I have placed all the myriad jar in the hadoop-yarn/lib directory and the classpath reflect that.
cp /tmp/myriad/myriad-scheduler/build/libs/* /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
cp /tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar /usr/libexec/mesos/
[root@nid00037 myriad]# su - yarn
-bash-4.1$ yarn classpath
/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-2557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/hadoop-yarn-client/lib/*
ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
-rw-r--r-- 1 root root 3456 Aug 19 08:50 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
-rw-r--r-- 1 root root 950687 Aug 19 08:50 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
and
-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
-rw-r--r-- 1 root root 39817 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
-rw-r--r-- 1 root root 192699 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
-rw-r--r-- 1 root root 229860 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
-rw-r--r-- 1 root root 232248 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
-rw-r--r-- 1 root root 1138921 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
-rw-r--r-- 1 root root 321751 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
-rw-r--r-- 1 root root 18336 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
-rw-r--r-- 1 root root 780664 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
-rw-r--r-- 1 root root 27084 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
-bash-4.1$ jar tf /usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.jar | grep YAMLFactory
com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
Re: myriad scheduler startup with HDP2.7
Posted by John Omernik <jo...@omernik.com>.
Fundamentally, I would imagine that the goals of Myriad, even using a
distribution's build of hadoop, is to have the classpath be entirely
contained. I.e. There should be no need for any classpath on a node to run
resource manager or node manager. This may post challenges, in that I know
in MapR some libs are linked to the /opt/mapr/lib folder. Perhaps when we
talk about building the tarball in the remote distribution, we should
explore this idea, and perhaps use flags that include the files if they are
links.
John
On Wed, Aug 19, 2015 at 1:53 PM, yuliya Feldman <yufeldman@yahoo.com.invalid
> wrote:
> as you can imagine you need matching versions of jackson* jars otherwise
> you might get into issues of incompatibility
> Easiest for you at the moment to put myriad dependency jars on classpath
> before others
> From: Bill Sparks <js...@cray.com>
> To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
> Cc: yuliya Feldman <yu...@yahoo.com>
> Sent: Wednesday, August 19, 2015 10:48 AM
> Subject: Re: myriad scheduler startup with HDP2.7
>
> Well thats the point, I do have 2.2.3 installed as that's the version
> shipped with HDP 2.3 and that gets loaded first in the classpath for YARN
> resourcemanager.
>
> I guess I have three alternatives.
>
> 1) build myriad using 2.2.3, thus matching the HDP installed jar's
> 2) replace the HDP version with 2.5.1, not sure what's that going to do
> for HDP compatibility
> 3) prepend a new classpath for yarn resourcemanager to pick up myriad
> versioned jars first.
>
> --
> Jonathan (Bill) Sparks
> Software Architecture
> Cray Inc.
>
>
>
>
>
>
>
> On 8/19/15 12:36 PM, "Adam Bordelon" <ad...@mesosphere.io> wrote:
>
> >Myriad should be using jackson 2.5.1
> >
> https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6
> >743/myriad-scheduler/build.gradle#L13
> >Double-check your build.gradle, and make sure you don't have a jackson
> >2.2.3 preinstalled somewhere else on your system
> >
> >On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks <js...@cray.com> wrote:
> >
> >> Odd the class path reported in the yarn log contains jackson-core-2.2.3
> >> and not 2.5.1. Is there a way to build myriad to match the version
> >> supported by HDP - that being 2.2.3 ?
> >>
> >>
> >> --
> >> Jonathan (Bill) Sparks
> >> Software Architecture
> >> Cray Inc.
> >>
> >>
> >>
> >>
> >>
> >> On 8/19/15 10:11 AM, "Bill Sparks" <js...@cray.com> wrote:
> >>
> >> >Thanks I'll check..
> >> >
> >> >--
> >> >Jonathan (Bill) Sparks
> >> >Software Architecture
> >> >Cray Inc.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID>
> >> wrote:
> >> >
> >> >>This method is part of JsonFactory class which is part of jackson-core
> >> >>jar
> >> >>See if you have some other jars on the classpath (different versions)
> >> >>that precede jackson-core-2.5.1.jar
> >> >> From: Bill Sparks <js...@cray.com>
> >> >> To: "dev@myriad.incubator.apache.org"
> >><de...@myriad.incubator.apache.org>
> >> >> Sent: Wednesday, August 19, 2015 7:08 AM
> >> >> Subject: myriad scheduler startup with HDP2.7
> >> >>
> >> >>I'm sure this is been resolved, but I've been triaging why I'm getting
> >> >>the following error on resourcemanager startup. Everything on the
> >> >>configuration side looks correct, but I must have missed something.
> >> >>
> >> >>
> >> >>
> >> >>2015-08-19 08:53:04,718 FATAL
> >> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> >> >>starting ResourceManager
> >> >>
> >> >>java.lang.NoSuchMethodError:
> >>
> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/In
> >>>>pu
> >> >>t
> >> >>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
> >> >>
> >> >>at
> >>
> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
> >>>>or
> >> >>y
> >> >>.java:299)
> >> >>
> >> >>at
> >>
> >>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
> >>>>or
> >> >>y
> >> >>.java:14)
> >> >>
> >> >>at
> >>
> >>>>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java
> >>>>:2
> >> >>0
> >> >>11)
> >> >>
> >> >>at com.ebay.myriad.Main.initialize(Main.java:70)
> >> >>
> >> >>at
> >>
> >>>>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercep
> >>>>to
> >> >>r
> >> >>.init(MyriadInitializationInterceptor.java:32)
> >> >>
> >> >>at
> >>
> >>>>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Co
> >>>>mp
> >> >>o
> >> >>siteInterceptor.java:76)
> >> >>
> >> >>at
> >>
> >>>>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFa
> >>>>ir
> >> >>S
> >> >>cheduler.java:50)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
> >>>>)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService
> >>>>.j
> >> >>a
> >> >>va:107)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveS
> >>>>er
> >> >>v
> >> >>ices.serviceInit(ResourceManager.java:572)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
> >>>>)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAnd
> >>>>In
> >> >>i
> >> >>tActiveServices(ResourceManager.java:972)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceIn
> >>>>it
> >> >>(
> >> >>ResourceManager.java:259)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
> >>>>)
> >> >>
> >> >>at
> >>
> >>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Reso
> >>>>ur
> >> >>c
> >> >>eManager.java:1202)
> >> >>
> >> >>
> >> >>I have placed all the myriad jar in the hadoop-yarn/lib directory and
> >>the
> >> >>classpath reflect that.
> >> >>
> >> >>
> >> >>cp /tmp/myriad/myriad-scheduler/build/libs/*
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
> >> >>
> >> >>cp
> >>
> >>>>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.j
> >>>>ar
> >> >>/usr/libexec/mesos/
> >> >>
> >> >>
> >> >>[root@nid00037 myriad]# su - yarn
> >> >>
> >> >>-bash-4.1$ yarn classpath
> >> >>
> >>
> >>>>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/us
> >>>>r/
> >> >>h
> >>
> >>>>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp
> >>>>/2
> >> >>.
> >>
> >>>>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.
> >>>>3.
> >> >>0
> >>
> >>>>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/h
> >>>>dp
> >> >>/
> >>
> >>>>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/
> >>>>us
> >> >>r
> >>
> >>>>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-m
> >>>>ap
> >> >>r
> >>
> >>>>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/
> >>>>ja
> >> >>v
> >>
> >>>>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/
> >>>>us
> >> >>r
> >>
> >>>>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/
> 2.3.0.
> >>>>0-
> >> >>2
> >>
> >>>>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/
> >>>>ha
> >> >>d
> >> >>oop-yarn-client/lib/*
> >> >>
> >> >>
> >> >>
> >> >>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
> >> >>
> >> >>
> >> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
> >> >>
> >> >>-rw-r--r-- 1 root root 3456 Aug 19 08:50
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 950687 Aug 19 08:50
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
> >> >>
> >> >>and
> >> >>
> >> >>
> >> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
> >> >>
> >> >>-rw-r--r-- 1 root root 39817 Aug 17 18:32
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 192699 Jul 14 08:22
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 229860 Aug 17 18:32
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 232248 Jul 14 08:22
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 321751 Aug 17 18:32
> >>
> >>>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 18336 Jul 14 08:22
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 780664 Jul 14 08:22
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
> >> >>
> >> >>-rw-r--r-- 1 root root 27084 Jul 14 08:22
> >> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
> >> >>
> >> >>
> >> >>
> >> >>-bash-4.1$ jar tf
> >>
> >>>>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.j
> >>>>ar
> >> >> | grep YAMLFactory
> >> >>
> >> >>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
> >> >>
> >> >>
> >> >>
> >> >
> >>
> >>
>
>
>
>
Re: myriad scheduler startup with HDP2.7
Posted by yuliya Feldman <yu...@yahoo.com.INVALID>.
as you can imagine you need matching versions of jackson* jars otherwise you might get into issues of incompatibility
Easiest for you at the moment to put myriad dependency jars on classpath before others
From: Bill Sparks <js...@cray.com>
To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
Cc: yuliya Feldman <yu...@yahoo.com>
Sent: Wednesday, August 19, 2015 10:48 AM
Subject: Re: myriad scheduler startup with HDP2.7
Well thats the point, I do have 2.2.3 installed as that's the version
shipped with HDP 2.3 and that gets loaded first in the classpath for YARN
resourcemanager.
I guess I have three alternatives.
1) build myriad using 2.2.3, thus matching the HDP installed jar's
2) replace the HDP version with 2.5.1, not sure what's that going to do
for HDP compatibility
3) prepend a new classpath for yarn resourcemanager to pick up myriad
versioned jars first.
--
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
On 8/19/15 12:36 PM, "Adam Bordelon" <ad...@mesosphere.io> wrote:
>Myriad should be using jackson 2.5.1
>https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6
>743/myriad-scheduler/build.gradle#L13
>Double-check your build.gradle, and make sure you don't have a jackson
>2.2.3 preinstalled somewhere else on your system
>
>On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks <js...@cray.com> wrote:
>
>> Odd the class path reported in the yarn log contains jackson-core-2.2.3
>> and not 2.5.1. Is there a way to build myriad to match the version
>> supported by HDP - that being 2.2.3 ?
>>
>>
>> --
>> Jonathan (Bill) Sparks
>> Software Architecture
>> Cray Inc.
>>
>>
>>
>>
>>
>> On 8/19/15 10:11 AM, "Bill Sparks" <js...@cray.com> wrote:
>>
>> >Thanks I'll check..
>> >
>> >--
>> >Jonathan (Bill) Sparks
>> >Software Architecture
>> >Cray Inc.
>> >
>> >
>> >
>> >
>> >
>> >On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID>
>> wrote:
>> >
>> >>This method is part of JsonFactory class which is part of jackson-core
>> >>jar
>> >>See if you have some other jars on the classpath (different versions)
>> >>that precede jackson-core-2.5.1.jar
>> >> From: Bill Sparks <js...@cray.com>
>> >> To: "dev@myriad.incubator.apache.org"
>><de...@myriad.incubator.apache.org>
>> >> Sent: Wednesday, August 19, 2015 7:08 AM
>> >> Subject: myriad scheduler startup with HDP2.7
>> >>
>> >>I'm sure this is been resolved, but I've been triaging why I'm getting
>> >>the following error on resourcemanager startup. Everything on the
>> >>configuration side looks correct, but I must have missed something.
>> >>
>> >>
>> >>
>> >>2015-08-19 08:53:04,718 FATAL
>> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> >>starting ResourceManager
>> >>
>> >>java.lang.NoSuchMethodError:
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/In
>>>>pu
>> >>t
>> >>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
>>>>or
>> >>y
>> >>.java:299)
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
>>>>or
>> >>y
>> >>.java:14)
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java
>>>>:2
>> >>0
>> >>11)
>> >>
>> >>at com.ebay.myriad.Main.initialize(Main.java:70)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercep
>>>>to
>> >>r
>> >>.init(MyriadInitializationInterceptor.java:32)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Co
>>>>mp
>> >>o
>> >>siteInterceptor.java:76)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFa
>>>>ir
>> >>S
>> >>cheduler.java:50)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService
>>>>.j
>> >>a
>> >>va:107)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveS
>>>>er
>> >>v
>> >>ices.serviceInit(ResourceManager.java:572)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAnd
>>>>In
>> >>i
>> >>tActiveServices(ResourceManager.java:972)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceIn
>>>>it
>> >>(
>> >>ResourceManager.java:259)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Reso
>>>>ur
>> >>c
>> >>eManager.java:1202)
>> >>
>> >>
>> >>I have placed all the myriad jar in the hadoop-yarn/lib directory and
>>the
>> >>classpath reflect that.
>> >>
>> >>
>> >>cp /tmp/myriad/myriad-scheduler/build/libs/*
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
>> >>
>> >>cp
>>
>>>>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.j
>>>>ar
>> >>/usr/libexec/mesos/
>> >>
>> >>
>> >>[root@nid00037 myriad]# su - yarn
>> >>
>> >>-bash-4.1$ yarn classpath
>> >>
>>
>>>>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/us
>>>>r/
>> >>h
>>
>>>>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp
>>>>/2
>> >>.
>>
>>>>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.
>>>>3.
>> >>0
>>
>>>>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/h
>>>>dp
>> >>/
>>
>>>>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/
>>>>us
>> >>r
>>
>>>>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-m
>>>>ap
>> >>r
>>
>>>>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/
>>>>ja
>> >>v
>>
>>>>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/
>>>>us
>> >>r
>>
>>>>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.
>>>>0-
>> >>2
>>
>>>>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/
>>>>ha
>> >>d
>> >>oop-yarn-client/lib/*
>> >>
>> >>
>> >>
>> >>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
>> >>
>> >>
>> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
>> >>
>> >>-rw-r--r-- 1 root root 3456 Aug 19 08:50
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 950687 Aug 19 08:50
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
>> >>
>> >>and
>> >>
>> >>
>> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
>> >>
>> >>-rw-r--r-- 1 root root 39817 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 192699 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
>> >>
>> >>-rw-r--r-- 1 root root 229860 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 232248 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 321751 Aug 17 18:32
>>
>>>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 18336 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 780664 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 27084 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
>> >>
>> >>
>> >>
>> >>-bash-4.1$ jar tf
>>
>>>>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.j
>>>>ar
>> >> | grep YAMLFactory
>> >>
>> >>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
>> >>
>> >>
>> >>
>> >
>>
>>
Re: myriad scheduler startup with HDP2.7
Posted by Bill Sparks <js...@cray.com>.
Well thats the point, I do have 2.2.3 installed as that's the version
shipped with HDP 2.3 and that gets loaded first in the classpath for YARN
resourcemanager.
I guess I have three alternatives.
1) build myriad using 2.2.3, thus matching the HDP installed jar's
2) replace the HDP version with 2.5.1, not sure what's that going to do
for HDP compatibility
3) prepend a new classpath for yarn resourcemanager to pick up myriad
versioned jars first.
--
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
On 8/19/15 12:36 PM, "Adam Bordelon" <ad...@mesosphere.io> wrote:
>Myriad should be using jackson 2.5.1
>https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6
>743/myriad-scheduler/build.gradle#L13
>Double-check your build.gradle, and make sure you don't have a jackson
>2.2.3 preinstalled somewhere else on your system
>
>On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks <js...@cray.com> wrote:
>
>> Odd the class path reported in the yarn log contains jackson-core-2.2.3
>> and not 2.5.1. Is there a way to build myriad to match the version
>> supported by HDP - that being 2.2.3 ?
>>
>>
>> --
>> Jonathan (Bill) Sparks
>> Software Architecture
>> Cray Inc.
>>
>>
>>
>>
>>
>> On 8/19/15 10:11 AM, "Bill Sparks" <js...@cray.com> wrote:
>>
>> >Thanks I'll check..
>> >
>> >--
>> >Jonathan (Bill) Sparks
>> >Software Architecture
>> >Cray Inc.
>> >
>> >
>> >
>> >
>> >
>> >On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID>
>> wrote:
>> >
>> >>This method is part of JsonFactory class which is part of jackson-core
>> >>jar
>> >>See if you have some other jars on the classpath (different versions)
>> >>that precede jackson-core-2.5.1.jar
>> >> From: Bill Sparks <js...@cray.com>
>> >> To: "dev@myriad.incubator.apache.org"
>><de...@myriad.incubator.apache.org>
>> >> Sent: Wednesday, August 19, 2015 7:08 AM
>> >> Subject: myriad scheduler startup with HDP2.7
>> >>
>> >>I'm sure this is been resolved, but I've been triaging why I'm getting
>> >>the following error on resourcemanager startup. Everything on the
>> >>configuration side looks correct, but I must have missed something.
>> >>
>> >>
>> >>
>> >>2015-08-19 08:53:04,718 FATAL
>> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>> >>starting ResourceManager
>> >>
>> >>java.lang.NoSuchMethodError:
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/In
>>>>pu
>> >>t
>> >>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
>>>>or
>> >>y
>> >>.java:299)
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFact
>>>>or
>> >>y
>> >>.java:14)
>> >>
>> >>at
>>
>>>>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java
>>>>:2
>> >>0
>> >>11)
>> >>
>> >>at com.ebay.myriad.Main.initialize(Main.java:70)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercep
>>>>to
>> >>r
>> >>.init(MyriadInitializationInterceptor.java:32)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Co
>>>>mp
>> >>o
>> >>siteInterceptor.java:76)
>> >>
>> >>at
>>
>>>>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFa
>>>>ir
>> >>S
>> >>cheduler.java:50)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService
>>>>.j
>> >>a
>> >>va:107)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveS
>>>>er
>> >>v
>> >>ices.serviceInit(ResourceManager.java:572)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAnd
>>>>In
>> >>i
>> >>tActiveServices(ResourceManager.java:972)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceIn
>>>>it
>> >>(
>> >>ResourceManager.java:259)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163
>>>>)
>> >>
>> >>at
>>
>>>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Reso
>>>>ur
>> >>c
>> >>eManager.java:1202)
>> >>
>> >>
>> >>I have placed all the myriad jar in the hadoop-yarn/lib directory and
>>the
>> >>classpath reflect that.
>> >>
>> >>
>> >>cp /tmp/myriad/myriad-scheduler/build/libs/*
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
>> >>
>> >>cp
>>
>>>>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.j
>>>>ar
>> >>/usr/libexec/mesos/
>> >>
>> >>
>> >>[root@nid00037 myriad]# su - yarn
>> >>
>> >>-bash-4.1$ yarn classpath
>> >>
>>
>>>>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/us
>>>>r/
>> >>h
>>
>>>>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp
>>>>/2
>> >>.
>>
>>>>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.
>>>>3.
>> >>0
>>
>>>>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/h
>>>>dp
>> >>/
>>
>>>>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/
>>>>us
>> >>r
>>
>>>>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-m
>>>>ap
>> >>r
>>
>>>>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/
>>>>ja
>> >>v
>>
>>>>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/
>>>>us
>> >>r
>>
>>>>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.
>>>>0-
>> >>2
>>
>>>>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/
>>>>ha
>> >>d
>> >>oop-yarn-client/lib/*
>> >>
>> >>
>> >>
>> >>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
>> >>
>> >>
>> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
>> >>
>> >>-rw-r--r-- 1 root root 3456 Aug 19 08:50
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 950687 Aug 19 08:50
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
>> >>
>> >>and
>> >>
>> >>
>> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
>> >>
>> >>-rw-r--r-- 1 root root 39817 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 192699 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
>> >>
>> >>-rw-r--r-- 1 root root 229860 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 232248 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 321751 Aug 17 18:32
>>
>>>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
>> >>
>> >>-rw-r--r-- 1 root root 18336 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 780664 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
>> >>
>> >>-rw-r--r-- 1 root root 27084 Jul 14 08:22
>> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
>> >>
>> >>
>> >>
>> >>-bash-4.1$ jar tf
>>
>>>>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.j
>>>>ar
>> >> | grep YAMLFactory
>> >>
>> >>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
>> >>
>> >>
>> >>
>> >
>>
>>
Re: myriad scheduler startup with HDP2.7
Posted by Adam Bordelon <ad...@mesosphere.io>.
Myriad should be using jackson 2.5.1
https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6743/myriad-scheduler/build.gradle#L13
Double-check your build.gradle, and make sure you don't have a jackson
2.2.3 preinstalled somewhere else on your system
On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks <js...@cray.com> wrote:
> Odd the class path reported in the yarn log contains jackson-core-2.2.3
> and not 2.5.1. Is there a way to build myriad to match the version
> supported by HDP - that being 2.2.3 ?
>
>
> --
> Jonathan (Bill) Sparks
> Software Architecture
> Cray Inc.
>
>
>
>
>
> On 8/19/15 10:11 AM, "Bill Sparks" <js...@cray.com> wrote:
>
> >Thanks I'll check..
> >
> >--
> >Jonathan (Bill) Sparks
> >Software Architecture
> >Cray Inc.
> >
> >
> >
> >
> >
> >On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID>
> wrote:
> >
> >>This method is part of JsonFactory class which is part of jackson-core
> >>jar
> >>See if you have some other jars on the classpath (different versions)
> >>that precede jackson-core-2.5.1.jar
> >> From: Bill Sparks <js...@cray.com>
> >> To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
> >> Sent: Wednesday, August 19, 2015 7:08 AM
> >> Subject: myriad scheduler startup with HDP2.7
> >>
> >>I'm sure this is been resolved, but I've been triaging why I'm getting
> >>the following error on resourcemanager startup. Everything on the
> >>configuration side looks correct, but I must have missed something.
> >>
> >>
> >>
> >>2015-08-19 08:53:04,718 FATAL
> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> >>starting ResourceManager
> >>
> >>java.lang.NoSuchMethodError:
> >>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/Inpu
> >>t
> >>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
> >>
> >>at
> >>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
> >>y
> >>.java:299)
> >>
> >>at
> >>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
> >>y
> >>.java:14)
> >>
> >>at
> >>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2
> >>0
> >>11)
> >>
> >>at com.ebay.myriad.Main.initialize(Main.java:70)
> >>
> >>at
> >>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercepto
> >>r
> >>.init(MyriadInitializationInterceptor.java:32)
> >>
> >>at
> >>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Comp
> >>o
> >>siteInterceptor.java:76)
> >>
> >>at
> >>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFair
> >>S
> >>cheduler.java:50)
> >>
> >>at
> >>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> >>
> >>at
> >>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.j
> >>a
> >>va:107)
> >>
> >>at
> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveSer
> >>v
> >>ices.serviceInit(ResourceManager.java:572)
> >>
> >>at
> >>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> >>
> >>at
> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndIn
> >>i
> >>tActiveServices(ResourceManager.java:972)
> >>
> >>at
> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit
> >>(
> >>ResourceManager.java:259)
> >>
> >>at
> >>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> >>
> >>at
> >>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Resour
> >>c
> >>eManager.java:1202)
> >>
> >>
> >>I have placed all the myriad jar in the hadoop-yarn/lib directory and the
> >>classpath reflect that.
> >>
> >>
> >>cp /tmp/myriad/myriad-scheduler/build/libs/*
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
> >>
> >>cp
> >>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar
> >>/usr/libexec/mesos/
> >>
> >>
> >>[root@nid00037 myriad]# su - yarn
> >>
> >>-bash-4.1$ yarn classpath
> >>
> >>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/
> >>h
> >>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2
> >>.
> >>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.
> >>0
> >>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp
> >>/
> >>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/us
> >>r
> >>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-map
> >>r
> >>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/ja
> >>v
> >>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/us
> >>r
> >>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-
> >>2
> >>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/ha
> >>d
> >>oop-yarn-client/lib/*
> >>
> >>
> >>
> >>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
> >>
> >>
> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
> >>
> >>-rw-r--r-- 1 root root 3456 Aug 19 08:50
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
> >>
> >>-rw-r--r-- 1 root root 950687 Aug 19 08:50
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
> >>
> >>and
> >>
> >>
> >>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
> >>
> >>-rw-r--r-- 1 root root 39817 Aug 17 18:32
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
> >>
> >>-rw-r--r-- 1 root root 192699 Jul 14 08:22
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
> >>
> >>-rw-r--r-- 1 root root 229860 Aug 17 18:32
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
> >>
> >>-rw-r--r-- 1 root root 232248 Jul 14 08:22
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
> >>
> >>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
> >>
> >>-rw-r--r-- 1 root root 321751 Aug 17 18:32
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
> >>
> >>-rw-r--r-- 1 root root 18336 Jul 14 08:22
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
> >>
> >>-rw-r--r-- 1 root root 780664 Jul 14 08:22
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
> >>
> >>-rw-r--r-- 1 root root 27084 Jul 14 08:22
> >>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
> >>
> >>
> >>
> >>-bash-4.1$ jar tf
> >>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.jar
> >> | grep YAMLFactory
> >>
> >>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
> >>
> >>
> >>
> >
>
>
Re: myriad scheduler startup with HDP2.7
Posted by Bill Sparks <js...@cray.com>.
Odd the class path reported in the yarn log contains jackson-core-2.2.3
and not 2.5.1. Is there a way to build myriad to match the version
supported by HDP - that being 2.2.3 ?
--
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
On 8/19/15 10:11 AM, "Bill Sparks" <js...@cray.com> wrote:
>Thanks I'll check..
>
>--
>Jonathan (Bill) Sparks
>Software Architecture
>Cray Inc.
>
>
>
>
>
>On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID> wrote:
>
>>This method is part of JsonFactory class which is part of jackson-core
>>jar
>>See if you have some other jars on the classpath (different versions)
>>that precede jackson-core-2.5.1.jar
>> From: Bill Sparks <js...@cray.com>
>> To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
>> Sent: Wednesday, August 19, 2015 7:08 AM
>> Subject: myriad scheduler startup with HDP2.7
>>
>>I'm sure this is been resolved, but I've been triaging why I'm getting
>>the following error on resourcemanager startup. Everything on the
>>configuration side looks correct, but I must have missed something.
>>
>>
>>
>>2015-08-19 08:53:04,718 FATAL
>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>starting ResourceManager
>>
>>java.lang.NoSuchMethodError:
>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/Inpu
>>t
>>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
>>
>>at
>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
>>y
>>.java:299)
>>
>>at
>>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
>>y
>>.java:14)
>>
>>at
>>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2
>>0
>>11)
>>
>>at com.ebay.myriad.Main.initialize(Main.java:70)
>>
>>at
>>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercepto
>>r
>>.init(MyriadInitializationInterceptor.java:32)
>>
>>at
>>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Comp
>>o
>>siteInterceptor.java:76)
>>
>>at
>>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFair
>>S
>>cheduler.java:50)
>>
>>at
>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>
>>at
>>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.j
>>a
>>va:107)
>>
>>at
>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveSer
>>v
>>ices.serviceInit(ResourceManager.java:572)
>>
>>at
>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>
>>at
>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndIn
>>i
>>tActiveServices(ResourceManager.java:972)
>>
>>at
>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit
>>(
>>ResourceManager.java:259)
>>
>>at
>>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>
>>at
>>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Resour
>>c
>>eManager.java:1202)
>>
>>
>>I have placed all the myriad jar in the hadoop-yarn/lib directory and the
>>classpath reflect that.
>>
>>
>>cp /tmp/myriad/myriad-scheduler/build/libs/*
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
>>
>>cp
>>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar
>>/usr/libexec/mesos/
>>
>>
>>[root@nid00037 myriad]# su - yarn
>>
>>-bash-4.1$ yarn classpath
>>
>>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/
>>h
>>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2
>>.
>>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.
>>0
>>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp
>>/
>>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/us
>>r
>>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-map
>>r
>>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/ja
>>v
>>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/us
>>r
>>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-
>>2
>>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/ha
>>d
>>oop-yarn-client/lib/*
>>
>>
>>
>>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
>>
>>
>>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
>>
>>-rw-r--r-- 1 root root 3456 Aug 19 08:50
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
>>
>>-rw-r--r-- 1 root root 950687 Aug 19 08:50
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
>>
>>and
>>
>>
>>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
>>
>>-rw-r--r-- 1 root root 39817 Aug 17 18:32
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
>>
>>-rw-r--r-- 1 root root 192699 Jul 14 08:22
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
>>
>>-rw-r--r-- 1 root root 229860 Aug 17 18:32
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
>>
>>-rw-r--r-- 1 root root 232248 Jul 14 08:22
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
>>
>>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
>>
>>-rw-r--r-- 1 root root 321751 Aug 17 18:32
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
>>
>>-rw-r--r-- 1 root root 18336 Jul 14 08:22
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
>>
>>-rw-r--r-- 1 root root 780664 Jul 14 08:22
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
>>
>>-rw-r--r-- 1 root root 27084 Jul 14 08:22
>>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
>>
>>
>>
>>-bash-4.1$ jar tf
>>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.jar
>> | grep YAMLFactory
>>
>>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
>>
>>
>>
>
Re: myriad scheduler startup with HDP2.7
Posted by Bill Sparks <js...@cray.com>.
Thanks I'll check..
--
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
On 8/19/15 10:09 AM, "yuliya Feldman" <yu...@yahoo.com.INVALID> wrote:
>This method is part of JsonFactory class which is part of jackson-core jar
>See if you have some other jars on the classpath (different versions)
>that precede jackson-core-2.5.1.jar
> From: Bill Sparks <js...@cray.com>
> To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
> Sent: Wednesday, August 19, 2015 7:08 AM
> Subject: myriad scheduler startup with HDP2.7
>
>I'm sure this is been resolved, but I've been triaging why I'm getting
>the following error on resourcemanager startup. Everything on the
>configuration side looks correct, but I must have missed something.
>
>
>
>2015-08-19 08:53:04,718 FATAL
>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>starting ResourceManager
>
>java.lang.NoSuchMethodError:
>com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/Input
>Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
>
>at
>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory
>.java:299)
>
>at
>com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory
>.java:14)
>
>at
>com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:20
>11)
>
>at com.ebay.myriad.Main.initialize(Main.java:70)
>
>at
>com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor
>.init(MyriadInitializationInterceptor.java:32)
>
>at
>com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Compo
>siteInterceptor.java:76)
>
>at
>com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairS
>cheduler.java:50)
>
>at
>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>
>at
>org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.ja
>va:107)
>
>at
>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServ
>ices.serviceInit(ResourceManager.java:572)
>
>at
>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>
>at
>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndIni
>tActiveServices(ResourceManager.java:972)
>
>at
>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(
>ResourceManager.java:259)
>
>at
>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>
>at
>org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Resourc
>eManager.java:1202)
>
>
>I have placed all the myriad jar in the hadoop-yarn/lib directory and the
>classpath reflect that.
>
>
>cp /tmp/myriad/myriad-scheduler/build/libs/*
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
>
>cp
>/tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar
>/usr/libexec/mesos/
>
>
>[root@nid00037 myriad]# su - yarn
>
>-bash-4.1$ yarn classpath
>
>/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/h
>dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2.
>3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.0
>.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp/
>2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/usr
>/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapr
>educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/jav
>a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/usr
>/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-2
>557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/had
>oop-yarn-client/lib/*
>
>
>
>ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
>
>
>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
>
>-rw-r--r-- 1 root root 3456 Aug 19 08:50
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
>
>-rw-r--r-- 1 root root 950687 Aug 19 08:50
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
>
>and
>
>
>-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
>
>-rw-r--r-- 1 root root 39817 Aug 17 18:32
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
>
>-rw-r--r-- 1 root root 192699 Jul 14 08:22
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
>
>-rw-r--r-- 1 root root 229860 Aug 17 18:32
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
>
>-rw-r--r-- 1 root root 232248 Jul 14 08:22
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
>
>-rw-r--r-- 1 root root 1138921 Aug 17 18:32
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
>
>-rw-r--r-- 1 root root 321751 Aug 17 18:32
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
>
>-rw-r--r-- 1 root root 18336 Jul 14 08:22
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
>
>-rw-r--r-- 1 root root 780664 Jul 14 08:22
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
>
>-rw-r--r-- 1 root root 27084 Jul 14 08:22
>/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
>
>
>
>-bash-4.1$ jar tf
>/usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.jar
> | grep YAMLFactory
>
>com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class
>
>
>
Re: myriad scheduler startup with HDP2.7
Posted by yuliya Feldman <yu...@yahoo.com.INVALID>.
This method is part of JsonFactory class which is part of jackson-core jar
See if you have some other jars on the classpath (different versions) that precede jackson-core-2.5.1.jar
From: Bill Sparks <js...@cray.com>
To: "dev@myriad.incubator.apache.org" <de...@myriad.incubator.apache.org>
Sent: Wednesday, August 19, 2015 7:08 AM
Subject: myriad scheduler startup with HDP2.7
I'm sure this is been resolved, but I've been triaging why I'm getting the following error on resourcemanager startup. Everything on the configuration side looks correct, but I must have missed something.
2015-08-19 08:53:04,718 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.NoSuchMethodError: com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/InputStream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:299)
at com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactory.java:14)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2011)
at com.ebay.myriad.Main.initialize(Main.java:70)
at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32)
at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76)
at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:572)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:972)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:259)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1202)
I have placed all the myriad jar in the hadoop-yarn/lib directory and the classpath reflect that.
cp /tmp/myriad/myriad-scheduler/build/libs/* /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
cp /tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar /usr/libexec/mesos/
[root@nid00037 myriad]# su - yarn
-bash-4.1$ yarn classpath
/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-mapreduce/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-2557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/hadoop-yarn-client/lib/*
ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
-rw-r--r-- 1 root root 3456 Aug 19 08:50 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
-rw-r--r-- 1 root root 950687 Aug 19 08:50 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
and
-bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
-rw-r--r-- 1 root root 39817 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
-rw-r--r-- 1 root root 192699 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.2.3.jar
-rw-r--r-- 1 root root 229860 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-2.5.1.jar
-rw-r--r-- 1 root root 232248 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar
-rw-r--r-- 1 root root 1138921 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-databind-2.5.1.jar
-rw-r--r-- 1 root root 321751 Aug 17 18:32 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-dataformat-yaml-2.5.1.jar
-rw-r--r-- 1 root root 18336 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar
-rw-r--r-- 1 root root 780664 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar
-rw-r--r-- 1 root root 27084 Jul 14 08:22 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-xc-1.9.13.jar
-bash-4.1$ jar tf /usr/hdp/current/hadoop-yarn-client/lib/jackson-dataformat-yaml-2.5.1.jar | grep YAMLFactory
com/fasterxml/jackson/dataformat/yaml/YAMLFactory.class