You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by "Nicolás Velásquez O." <gn...@gmail.com> on 2017/02/17 11:32:42 UTC
MapR problem. No FileSystem for scheme: maprfs
Hi,
I'm having problems to run Solr using Apache Slider on MapR. Specifically,
when I try to install the slider package it fails because slider doesn't
have support for maprfs:
$ slider install-package --replacepkg --name solr --package
~/inst/solr/solr-slider/solr-on-yarn.zip
2017-02-17 12:08:28,635 [main] INFO service.AbstractService - Service
Slider Client failed in state INITED; cause: java.io.IOException: No
FileSystem for scheme: maprfs
java.io.IOException: No FileSystem for scheme: maprfs
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
at
org.apache.slider.common.tools.CoreFileSystem.<init>(CoreFileSystem.java:82)
at
org.apache.slider.common.tools.SliderFileSystem.<init>(SliderFileSystem.java:38)
at
org.apache.slider.client.SliderClient.initHadoopBinding(SliderClient.java:498)
at
org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:182)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
at org.apache.slider.Slider.main(Slider.java:49)
Exception: java.io.IOException: No FileSystem for scheme: maprfs
2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception:
java.io.IOException: No FileSystem for scheme: maprfs
org.apache.hadoop.service.ServiceStateException: java.io.IOException: No
FileSystem for scheme: maprfs
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:182)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
at org.apache.slider.Slider.main(Slider.java:49)
Caused by: java.io.IOException: No FileSystem for scheme: maprfs
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
at
org.apache.slider.common.tools.CoreFileSystem.<init>(CoreFileSystem.java:82)
at
org.apache.slider.common.tools.SliderFileSystem.<init>(SliderFileSystem.java:38)
at
org.apache.slider.client.SliderClient.initHadoopBinding(SliderClient.java:498)
at
org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
... 5 more
2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with status 56
I have tried (with no success) to copy the maprfs jar to the slider lib
path, and to set the CLASSPATH on slider-env.sh as:
export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
and
export CLASSPATH=`hadoop classpath`
Has anybody got apache slider to work on top of mapr?
Thanks!
Nicolas
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by Gour Saha <gs...@hortonworks.com>.
Great, it worked for you.
The reason slider client did not pick up the change in
/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml is probably because
HADOOP_CONF_DIR in slider-env.sh is not set to that path. It is ok to set
that property in slider-client.xml which made the client submit the app,
but remember even the Slider AM would need that value when it comes up. So
you might have to add it to slider-server.xml as well.
I usually set HADOOP_CONF_DIR in slider-env.sh and that takes care of both.
Copying the 3 jars to slider lib dir is also the ok thing to do.
-Gour
On 2/17/17, 1:08 PM, "Nicolás Velásquez O." <gn...@gmail.com> wrote:
>By the way, I didn't modify the classpath in slider-env.sh. Instead I
>copied three jars from the MapR dist: maprfs-5.2.0-mapr.jar,
>hadoop-common-2.7.0.jar and json-20080701.jar
>
>
>
>On Fri, Feb 17, 2017 at 10:07 PM, Nicolás Velásquez O.
><gn...@gmail.com>
>wrote:
>
>> Hi Gour!
>> Thanks for pointing me in the right direction!! First I tried to
>>configure
>> "fs.maprfs.impl" on
>>/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml,
>> but that didn't work. I hoped Slider would read that core-site file,
>>but it
>> was not the case. However, since it was slider's client failing and the
>> cluster is MapR, I simply added that property to slider-client.xml and
>>it
>> worked!
>> Ps: I'm still getting some errors, but something unrelated to maprfs.
>> Cheers!
>> Nicolas
>>
>> On Fri, Feb 17, 2017 at 8:39 PM, Gour Saha <gs...@hortonworks.com>
>>wrote:
>>
>>> Can you set the config property "fs.maprfs.impl" to the MapR
>>> implementation class of org.apache.hadoop.fs.FileSystem and try again?
>>>
>>> I think you can set this property in any of the cluster site xmls,
>>> preferably core-site.xml.
>>>
>>> -Gour
>>>
>>> On 2/17/17, 11:14 AM, "jonmaron@gmail.com" <jo...@gmail.com> wrote:
>>>
>>> >My guess is there's actually a need to have the MapR hadoop common jar
>>> >(the
>>> >one I would assume has an updated file system implementation) be
>>> >pre-pended
>>> >to the class path. I don't have an installation in front of me, but
>>>I'd
>>> >look into modifying slider-env or the like to prepend at least that
>>>one
>>> >jar
>>> >to the class path.
>>> >
>>> >On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
>>> ><gn...@gmail.com>
>>> >wrote:
>>> >
>>> >> Hey,
>>> >> It's great to initiate MapR to slider!
>>> >> Unfortunately, following the suggestions did not work (remove the
>>> >>classpath
>>> >> modification on slider-env.sh and copied mapr's jars to slider's lib
>>> >> folder).
>>> >>
>>> >> I'm guessing support for parsing the fs.defaultFS string
>>>("maprfs:///")
>>> >>has
>>> >> to be added to Slider's code. Something similar to
>>> >> https://issues.apache.org/jira/browse/FLINK-1006.
>>> >>
>>> >> Should I just open an incident on Jira?
>>> >>
>>> >> Thanks!
>>> >> Nicolas
>>> >>
>>> >>
>>> >>
>>> >> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
>>> >><bi...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > I don't think anyone has tried running Slider on MapR yet, so you
>>>are
>>> >> > breaking new ground. I would not try to change CLASSPATH in
>>> >> slider-env.sh.
>>> >> > Slider has a standalone lib directory, meaning that all of its
>>> >>required
>>> >> > jars are in the lib dir (including hadoop jars). It might be the
>>>case
>>> >> that
>>> >> > more jars are needed from the mapr lib directory. I would try
>>>copying
>>> >>all
>>> >> > the jars from the mapr lib directory to the slider lib directory
>>>and
>>> >>see
>>> >> if
>>> >> > that makes a difference. If it works, then you might be able to
>>> remove
>>> >> some
>>> >> > of the jars.
>>> >> >
>>> >> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
>>> >> gnicolax@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> > > Hi,
>>> >> > >
>>> >> > > I'm having problems to run Solr using Apache Slider on MapR.
>>> >> > Specifically,
>>> >> > > when I try to install the slider package it fails because slider
>>> >> doesn't
>>> >> > > have support for maprfs:
>>> >> > > $ slider install-package --replacepkg --name solr --package
>>> >> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
>>> >> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
>>> >>Service
>>> >> > > Slider Client failed in state INITED; cause:
>>>java.io.IOException:
>>> No
>>> >> > > FileSystem for scheme: maprfs
>>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>>> >> FileSystem.java:2644)
>>> >> > > at
>>> >> > >
>>>
>>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>>> >> > FileSystem.java:92)
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>>> >> FileSystem.java:2687)
>>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>>> >> > FileSystem.java:2669)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:371)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:170)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>>> >> > > CoreFileSystem.java:82)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>>> >> > > SliderFileSystem.java:38)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>>> >> > > SliderClient.java:498)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>>> >> SliderClient.java:318)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:163)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>>> >> > > ServiceLauncher.java:182)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>>> >> > > ServiceLauncher.java:475)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>>> >> > > ServiceLauncher.java:403)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>>> >> > > ServiceLauncher.java:630)
>>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>>> >> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
>>> >>Exception:
>>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > org.apache.hadoop.service.ServiceStateException:
>>> >>java.io.IOException:
>>> >> No
>>> >> > > FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.service.ServiceStateException.convert(
>>> >> > > ServiceStateException.java:59)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:172)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>>> >> > > ServiceLauncher.java:182)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>>> >> > > ServiceLauncher.java:475)
>>> >> > > at
>>> >> > >
>>>org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>>> >> > > ServiceLauncher.java:403)
>>> >> > > at
>>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>>> >> > > ServiceLauncher.java:630)
>>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>>> >> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>>> >> FileSystem.java:2644)
>>> >> > > at
>>> >> > >
>>>
>>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>>> >> > FileSystem.java:92)
>>> >> > > at
>>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>>> >> FileSystem.java:2687)
>>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>>> >> > FileSystem.java:2669)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:371)
>>> >> > > at org.apache.hadoop.fs.FileSyste
>>> m.get(FileSystem.java:170)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>>> >> > > CoreFileSystem.java:82)
>>> >> > > at
>>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>>> >> > > SliderFileSystem.java:38)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>>> >> > > SliderClient.java:498)
>>> >> > > at
>>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>>> >> SliderClient.java:318)
>>> >> > > at
>>> >> > > org.apache.hadoop.service.AbstractService.init(
>>> >> AbstractService.java:163)
>>> >> > > ... 5 more
>>> >> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting
>>>with
>>> >> status
>>> >> > 56
>>> >> > >
>>> >> > > I have tried (with no success) to copy the maprfs jar to the
>>>slider
>>> >>lib
>>> >> > > path, and to set the CLASSPATH on slider-env.sh as:
>>> >> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
>>> >> > > and
>>> >> > > export CLASSPATH=`hadoop classpath`
>>> >> > >
>>> >> > > Has anybody got apache slider to work on top of mapr?
>>> >> > >
>>> >> > > Thanks!
>>> >> > > Nicolas
>>> >> > >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Nicolás Velásquez O.
>>> >> Mobile +41.767189348 <076%20718%2093%2048>
>>> >>
>>> >
>>> >
>>> >
>>> >--
>>> >Sent from some random computer....
>>>
>>>
>>
>>
>> --
>> Nicolás Velásquez O.
>> Mobile +41.767189348 <076%20718%2093%2048>
>>
>
>
>
>--
>Nicolás Velásquez O.
>Mobile +41.767189348
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by "Nicolás Velásquez O." <gn...@gmail.com>.
By the way, I didn't modify the classpath in slider-env.sh. Instead I
copied three jars from the MapR dist: maprfs-5.2.0-mapr.jar,
hadoop-common-2.7.0.jar and json-20080701.jar
On Fri, Feb 17, 2017 at 10:07 PM, Nicolás Velásquez O. <gn...@gmail.com>
wrote:
> Hi Gour!
> Thanks for pointing me in the right direction!! First I tried to configure
> "fs.maprfs.impl" on /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml,
> but that didn't work. I hoped Slider would read that core-site file, but it
> was not the case. However, since it was slider's client failing and the
> cluster is MapR, I simply added that property to slider-client.xml and it
> worked!
> Ps: I'm still getting some errors, but something unrelated to maprfs.
> Cheers!
> Nicolas
>
> On Fri, Feb 17, 2017 at 8:39 PM, Gour Saha <gs...@hortonworks.com> wrote:
>
>> Can you set the config property "fs.maprfs.impl" to the MapR
>> implementation class of org.apache.hadoop.fs.FileSystem and try again?
>>
>> I think you can set this property in any of the cluster site xmls,
>> preferably core-site.xml.
>>
>> -Gour
>>
>> On 2/17/17, 11:14 AM, "jonmaron@gmail.com" <jo...@gmail.com> wrote:
>>
>> >My guess is there's actually a need to have the MapR hadoop common jar
>> >(the
>> >one I would assume has an updated file system implementation) be
>> >pre-pended
>> >to the class path. I don't have an installation in front of me, but I'd
>> >look into modifying slider-env or the like to prepend at least that one
>> >jar
>> >to the class path.
>> >
>> >On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
>> ><gn...@gmail.com>
>> >wrote:
>> >
>> >> Hey,
>> >> It's great to initiate MapR to slider!
>> >> Unfortunately, following the suggestions did not work (remove the
>> >>classpath
>> >> modification on slider-env.sh and copied mapr's jars to slider's lib
>> >> folder).
>> >>
>> >> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///")
>> >>has
>> >> to be added to Slider's code. Something similar to
>> >> https://issues.apache.org/jira/browse/FLINK-1006.
>> >>
>> >> Should I just open an incident on Jira?
>> >>
>> >> Thanks!
>> >> Nicolas
>> >>
>> >>
>> >>
>> >> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
>> >><bi...@gmail.com>
>> >> wrote:
>> >>
>> >> > I don't think anyone has tried running Slider on MapR yet, so you are
>> >> > breaking new ground. I would not try to change CLASSPATH in
>> >> slider-env.sh.
>> >> > Slider has a standalone lib directory, meaning that all of its
>> >>required
>> >> > jars are in the lib dir (including hadoop jars). It might be the case
>> >> that
>> >> > more jars are needed from the mapr lib directory. I would try copying
>> >>all
>> >> > the jars from the mapr lib directory to the slider lib directory and
>> >>see
>> >> if
>> >> > that makes a difference. If it works, then you might be able to
>> remove
>> >> some
>> >> > of the jars.
>> >> >
>> >> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
>> >> gnicolax@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Hi,
>> >> > >
>> >> > > I'm having problems to run Solr using Apache Slider on MapR.
>> >> > Specifically,
>> >> > > when I try to install the slider package it fails because slider
>> >> doesn't
>> >> > > have support for maprfs:
>> >> > > $ slider install-package --replacepkg --name solr --package
>> >> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
>> >> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
>> >>Service
>> >> > > Slider Client failed in state INITED; cause: java.io.IOException:
>> No
>> >> > > FileSystem for scheme: maprfs
>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> >> FileSystem.java:2644)
>> >> > > at
>> >> > >
>> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> >> > FileSystem.java:92)
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> >> FileSystem.java:2687)
>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> >> > FileSystem.java:2669)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:371)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:170)
>> >> > > at
>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> >> > > CoreFileSystem.java:82)
>> >> > > at
>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> >> > > SliderFileSystem.java:38)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> >> > > SliderClient.java:498)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>> >> SliderClient.java:318)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:163)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> >> > > ServiceLauncher.java:182)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> >> > > ServiceLauncher.java:475)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> >> > > ServiceLauncher.java:403)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> >> > > ServiceLauncher.java:630)
>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>> >> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
>> >>Exception:
>> >> > > java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > org.apache.hadoop.service.ServiceStateException:
>> >>java.io.IOException:
>> >> No
>> >> > > FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.service.ServiceStateException.convert(
>> >> > > ServiceStateException.java:59)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:172)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> >> > > ServiceLauncher.java:182)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> >> > > ServiceLauncher.java:475)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> >> > > ServiceLauncher.java:403)
>> >> > > at
>> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> >> > > ServiceLauncher.java:630)
>> >> > > at org.apache.slider.Slider.main(Slider.java:49)
>> >> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> >> FileSystem.java:2644)
>> >> > > at
>> >> > >
>> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> >> > FileSystem.java:92)
>> >> > > at
>> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> >> FileSystem.java:2687)
>> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> >> > FileSystem.java:2669)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:371)
>> >> > > at org.apache.hadoop.fs.FileSyste
>> m.get(FileSystem.java:170)
>> >> > > at
>> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> >> > > CoreFileSystem.java:82)
>> >> > > at
>> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> >> > > SliderFileSystem.java:38)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> >> > > SliderClient.java:498)
>> >> > > at
>> >> > > org.apache.slider.client.SliderClient.serviceInit(
>> >> SliderClient.java:318)
>> >> > > at
>> >> > > org.apache.hadoop.service.AbstractService.init(
>> >> AbstractService.java:163)
>> >> > > ... 5 more
>> >> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with
>> >> status
>> >> > 56
>> >> > >
>> >> > > I have tried (with no success) to copy the maprfs jar to the slider
>> >>lib
>> >> > > path, and to set the CLASSPATH on slider-env.sh as:
>> >> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
>> >> > > and
>> >> > > export CLASSPATH=`hadoop classpath`
>> >> > >
>> >> > > Has anybody got apache slider to work on top of mapr?
>> >> > >
>> >> > > Thanks!
>> >> > > Nicolas
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Nicolás Velásquez O.
>> >> Mobile +41.767189348 <076%20718%2093%2048>
>> >>
>> >
>> >
>> >
>> >--
>> >Sent from some random computer....
>>
>>
>
>
> --
> Nicolás Velásquez O.
> Mobile +41.767189348 <076%20718%2093%2048>
>
--
Nicolás Velásquez O.
Mobile +41.767189348
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by "Nicolás Velásquez O." <gn...@gmail.com>.
Hi Gour!
Thanks for pointing me in the right direction!! First I tried to configure
"fs.maprfs.impl" on /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml,
but that didn't work. I hoped Slider would read that core-site file, but it
was not the case. However, since it was slider's client failing and the
cluster is MapR, I simply added that property to slider-client.xml and it
worked!
Ps: I'm still getting some errors, but something unrelated to maprfs.
Cheers!
Nicolas
On Fri, Feb 17, 2017 at 8:39 PM, Gour Saha <gs...@hortonworks.com> wrote:
> Can you set the config property "fs.maprfs.impl" to the MapR
> implementation class of org.apache.hadoop.fs.FileSystem and try again?
>
> I think you can set this property in any of the cluster site xmls,
> preferably core-site.xml.
>
> -Gour
>
> On 2/17/17, 11:14 AM, "jonmaron@gmail.com" <jo...@gmail.com> wrote:
>
> >My guess is there's actually a need to have the MapR hadoop common jar
> >(the
> >one I would assume has an updated file system implementation) be
> >pre-pended
> >to the class path. I don't have an installation in front of me, but I'd
> >look into modifying slider-env or the like to prepend at least that one
> >jar
> >to the class path.
> >
> >On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
> ><gn...@gmail.com>
> >wrote:
> >
> >> Hey,
> >> It's great to initiate MapR to slider!
> >> Unfortunately, following the suggestions did not work (remove the
> >>classpath
> >> modification on slider-env.sh and copied mapr's jars to slider's lib
> >> folder).
> >>
> >> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///")
> >>has
> >> to be added to Slider's code. Something similar to
> >> https://issues.apache.org/jira/browse/FLINK-1006.
> >>
> >> Should I just open an incident on Jira?
> >>
> >> Thanks!
> >> Nicolas
> >>
> >>
> >>
> >> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
> >><bi...@gmail.com>
> >> wrote:
> >>
> >> > I don't think anyone has tried running Slider on MapR yet, so you are
> >> > breaking new ground. I would not try to change CLASSPATH in
> >> slider-env.sh.
> >> > Slider has a standalone lib directory, meaning that all of its
> >>required
> >> > jars are in the lib dir (including hadoop jars). It might be the case
> >> that
> >> > more jars are needed from the mapr lib directory. I would try copying
> >>all
> >> > the jars from the mapr lib directory to the slider lib directory and
> >>see
> >> if
> >> > that makes a difference. If it works, then you might be able to remove
> >> some
> >> > of the jars.
> >> >
> >> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
> >> gnicolax@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I'm having problems to run Solr using Apache Slider on MapR.
> >> > Specifically,
> >> > > when I try to install the slider package it fails because slider
> >> doesn't
> >> > > have support for maprfs:
> >> > > $ slider install-package --replacepkg --name solr --package
> >> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
> >> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
> >>Service
> >> > > Slider Client failed in state INITED; cause: java.io.IOException: No
> >> > > FileSystem for scheme: maprfs
> >> > > java.io.IOException: No FileSystem for scheme: maprfs
> >> > > at
> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> >> FileSystem.java:2644)
> >> > > at
> >> > >
> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
> >> > FileSystem.java:92)
> >> > > at
> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> >> FileSystem.java:2687)
> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
> >> > FileSystem.java:2669)
> >> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> >> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> >> > > at
> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
> >> > > CoreFileSystem.java:82)
> >> > > at
> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
> >> > > SliderFileSystem.java:38)
> >> > > at
> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
> >> > > SliderClient.java:498)
> >> > > at
> >> > > org.apache.slider.client.SliderClient.serviceInit(
> >> SliderClient.java:318)
> >> > > at
> >> > > org.apache.hadoop.service.AbstractService.init(
> >> AbstractService.java:163)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> >> > > ServiceLauncher.java:182)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> >> > > ServiceLauncher.java:475)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> >> > > ServiceLauncher.java:403)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> >> > > ServiceLauncher.java:630)
> >> > > at org.apache.slider.Slider.main(Slider.java:49)
> >> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
> >> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
> >>Exception:
> >> > > java.io.IOException: No FileSystem for scheme: maprfs
> >> > > org.apache.hadoop.service.ServiceStateException:
> >>java.io.IOException:
> >> No
> >> > > FileSystem for scheme: maprfs
> >> > > at
> >> > > org.apache.hadoop.service.ServiceStateException.convert(
> >> > > ServiceStateException.java:59)
> >> > > at
> >> > > org.apache.hadoop.service.AbstractService.init(
> >> AbstractService.java:172)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> >> > > ServiceLauncher.java:182)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> >> > > ServiceLauncher.java:475)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> >> > > ServiceLauncher.java:403)
> >> > > at
> >> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> >> > > ServiceLauncher.java:630)
> >> > > at org.apache.slider.Slider.main(Slider.java:49)
> >> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
> >> > > at
> >> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> >> FileSystem.java:2644)
> >> > > at
> >> > >
> >>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> >> > > at org.apache.hadoop.fs.FileSystem.access$200(
> >> > FileSystem.java:92)
> >> > > at
> >> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> >> FileSystem.java:2687)
> >> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
> >> > FileSystem.java:2669)
> >> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> >> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> >> > > at
> >> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
> >> > > CoreFileSystem.java:82)
> >> > > at
> >> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
> >> > > SliderFileSystem.java:38)
> >> > > at
> >> > > org.apache.slider.client.SliderClient.initHadoopBinding(
> >> > > SliderClient.java:498)
> >> > > at
> >> > > org.apache.slider.client.SliderClient.serviceInit(
> >> SliderClient.java:318)
> >> > > at
> >> > > org.apache.hadoop.service.AbstractService.init(
> >> AbstractService.java:163)
> >> > > ... 5 more
> >> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with
> >> status
> >> > 56
> >> > >
> >> > > I have tried (with no success) to copy the maprfs jar to the slider
> >>lib
> >> > > path, and to set the CLASSPATH on slider-env.sh as:
> >> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
> >> > > and
> >> > > export CLASSPATH=`hadoop classpath`
> >> > >
> >> > > Has anybody got apache slider to work on top of mapr?
> >> > >
> >> > > Thanks!
> >> > > Nicolas
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Nicolás Velásquez O.
> >> Mobile +41.767189348
> >>
> >
> >
> >
> >--
> >Sent from some random computer....
>
>
--
Nicolás Velásquez O.
Mobile +41.767189348
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by Gour Saha <gs...@hortonworks.com>.
Can you set the config property "fs.maprfs.impl" to the MapR
implementation class of org.apache.hadoop.fs.FileSystem and try again?
I think you can set this property in any of the cluster site xmls,
preferably core-site.xml.
-Gour
On 2/17/17, 11:14 AM, "jonmaron@gmail.com" <jo...@gmail.com> wrote:
>My guess is there's actually a need to have the MapR hadoop common jar
>(the
>one I would assume has an updated file system implementation) be
>pre-pended
>to the class path. I don't have an installation in front of me, but I'd
>look into modifying slider-env or the like to prepend at least that one
>jar
>to the class path.
>
>On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O.
><gn...@gmail.com>
>wrote:
>
>> Hey,
>> It's great to initiate MapR to slider!
>> Unfortunately, following the suggestions did not work (remove the
>>classpath
>> modification on slider-env.sh and copied mapr's jars to slider's lib
>> folder).
>>
>> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///")
>>has
>> to be added to Slider's code. Something similar to
>> https://issues.apache.org/jira/browse/FLINK-1006.
>>
>> Should I just open an incident on Jira?
>>
>> Thanks!
>> Nicolas
>>
>>
>>
>> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi
>><bi...@gmail.com>
>> wrote:
>>
>> > I don't think anyone has tried running Slider on MapR yet, so you are
>> > breaking new ground. I would not try to change CLASSPATH in
>> slider-env.sh.
>> > Slider has a standalone lib directory, meaning that all of its
>>required
>> > jars are in the lib dir (including hadoop jars). It might be the case
>> that
>> > more jars are needed from the mapr lib directory. I would try copying
>>all
>> > the jars from the mapr lib directory to the slider lib directory and
>>see
>> if
>> > that makes a difference. If it works, then you might be able to remove
>> some
>> > of the jars.
>> >
>> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
>> gnicolax@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I'm having problems to run Solr using Apache Slider on MapR.
>> > Specifically,
>> > > when I try to install the slider package it fails because slider
>> doesn't
>> > > have support for maprfs:
>> > > $ slider install-package --replacepkg --name solr --package
>> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
>> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService -
>>Service
>> > > Slider Client failed in state INITED; cause: java.io.IOException: No
>> > > FileSystem for scheme: maprfs
>> > > java.io.IOException: No FileSystem for scheme: maprfs
>> > > at
>> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> FileSystem.java:2644)
>> > > at
>> > >
>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> > FileSystem.java:92)
>> > > at
>> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> FileSystem.java:2687)
>> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> > FileSystem.java:2669)
>> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
>> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
>> > > at
>> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> > > CoreFileSystem.java:82)
>> > > at
>> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> > > SliderFileSystem.java:38)
>> > > at
>> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> > > SliderClient.java:498)
>> > > at
>> > > org.apache.slider.client.SliderClient.serviceInit(
>> SliderClient.java:318)
>> > > at
>> > > org.apache.hadoop.service.AbstractService.init(
>> AbstractService.java:163)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> > > ServiceLauncher.java:182)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> > > ServiceLauncher.java:475)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> > > ServiceLauncher.java:403)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> > > ServiceLauncher.java:630)
>> > > at org.apache.slider.Slider.main(Slider.java:49)
>> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
>> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher -
>>Exception:
>> > > java.io.IOException: No FileSystem for scheme: maprfs
>> > > org.apache.hadoop.service.ServiceStateException:
>>java.io.IOException:
>> No
>> > > FileSystem for scheme: maprfs
>> > > at
>> > > org.apache.hadoop.service.ServiceStateException.convert(
>> > > ServiceStateException.java:59)
>> > > at
>> > > org.apache.hadoop.service.AbstractService.init(
>> AbstractService.java:172)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchService(
>> > > ServiceLauncher.java:182)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
>> > > ServiceLauncher.java:475)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
>> > > ServiceLauncher.java:403)
>> > > at
>> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
>> > > ServiceLauncher.java:630)
>> > > at org.apache.slider.Slider.main(Slider.java:49)
>> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
>> > > at
>> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
>> FileSystem.java:2644)
>> > > at
>> > >
>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
>> > > at org.apache.hadoop.fs.FileSystem.access$200(
>> > FileSystem.java:92)
>> > > at
>> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
>> FileSystem.java:2687)
>> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
>> > FileSystem.java:2669)
>> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
>> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
>> > > at
>> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
>> > > CoreFileSystem.java:82)
>> > > at
>> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
>> > > SliderFileSystem.java:38)
>> > > at
>> > > org.apache.slider.client.SliderClient.initHadoopBinding(
>> > > SliderClient.java:498)
>> > > at
>> > > org.apache.slider.client.SliderClient.serviceInit(
>> SliderClient.java:318)
>> > > at
>> > > org.apache.hadoop.service.AbstractService.init(
>> AbstractService.java:163)
>> > > ... 5 more
>> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with
>> status
>> > 56
>> > >
>> > > I have tried (with no success) to copy the maprfs jar to the slider
>>lib
>> > > path, and to set the CLASSPATH on slider-env.sh as:
>> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
>> > > and
>> > > export CLASSPATH=`hadoop classpath`
>> > >
>> > > Has anybody got apache slider to work on top of mapr?
>> > >
>> > > Thanks!
>> > > Nicolas
>> > >
>> >
>>
>>
>>
>> --
>> Nicolás Velásquez O.
>> Mobile +41.767189348
>>
>
>
>
>--
>Sent from some random computer....
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by "jonmaron@gmail.com" <jo...@gmail.com>.
My guess is there's actually a need to have the MapR hadoop common jar (the
one I would assume has an updated file system implementation) be pre-pended
to the class path. I don't have an installation in front of me, but I'd
look into modifying slider-env or the like to prepend at least that one jar
to the class path.
On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O. <gn...@gmail.com>
wrote:
> Hey,
> It's great to initiate MapR to slider!
> Unfortunately, following the suggestions did not work (remove the classpath
> modification on slider-env.sh and copied mapr's jars to slider's lib
> folder).
>
> I'm guessing support for parsing the fs.defaultFS string ("maprfs:///") has
> to be added to Slider's code. Something similar to
> https://issues.apache.org/jira/browse/FLINK-1006.
>
> Should I just open an incident on Jira?
>
> Thanks!
> Nicolas
>
>
>
> On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi <bi...@gmail.com>
> wrote:
>
> > I don't think anyone has tried running Slider on MapR yet, so you are
> > breaking new ground. I would not try to change CLASSPATH in
> slider-env.sh.
> > Slider has a standalone lib directory, meaning that all of its required
> > jars are in the lib dir (including hadoop jars). It might be the case
> that
> > more jars are needed from the mapr lib directory. I would try copying all
> > the jars from the mapr lib directory to the slider lib directory and see
> if
> > that makes a difference. If it works, then you might be able to remove
> some
> > of the jars.
> >
> > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <
> gnicolax@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm having problems to run Solr using Apache Slider on MapR.
> > Specifically,
> > > when I try to install the slider package it fails because slider
> doesn't
> > > have support for maprfs:
> > > $ slider install-package --replacepkg --name solr --package
> > > ~/inst/solr/solr-slider/solr-on-yarn.zip
> > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService - Service
> > > Slider Client failed in state INITED; cause: java.io.IOException: No
> > > FileSystem for scheme: maprfs
> > > java.io.IOException: No FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> FileSystem.java:2644)
> > > at
> > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> > > at org.apache.hadoop.fs.FileSystem.access$200(
> > FileSystem.java:92)
> > > at
> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> FileSystem.java:2687)
> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
> > FileSystem.java:2669)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> > > at
> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
> > > CoreFileSystem.java:82)
> > > at
> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
> > > SliderFileSystem.java:38)
> > > at
> > > org.apache.slider.client.SliderClient.initHadoopBinding(
> > > SliderClient.java:498)
> > > at
> > > org.apache.slider.client.SliderClient.serviceInit(
> SliderClient.java:318)
> > > at
> > > org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:163)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> > > ServiceLauncher.java:182)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > > ServiceLauncher.java:475)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > > ServiceLauncher.java:403)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > > ServiceLauncher.java:630)
> > > at org.apache.slider.Slider.main(Slider.java:49)
> > > Exception: java.io.IOException: No FileSystem for scheme: maprfs
> > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception:
> > > java.io.IOException: No FileSystem for scheme: maprfs
> > > org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> No
> > > FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.service.ServiceStateException.convert(
> > > ServiceStateException.java:59)
> > > at
> > > org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:172)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchService(
> > > ServiceLauncher.java:182)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > > ServiceLauncher.java:475)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > > ServiceLauncher.java:403)
> > > at
> > > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > > ServiceLauncher.java:630)
> > > at org.apache.slider.Slider.main(Slider.java:49)
> > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
> > > at
> > > org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> FileSystem.java:2644)
> > > at
> > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> > > at org.apache.hadoop.fs.FileSystem.access$200(
> > FileSystem.java:92)
> > > at
> > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> FileSystem.java:2687)
> > > at org.apache.hadoop.fs.FileSystem$Cache.get(
> > FileSystem.java:2669)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> > > at
> > > org.apache.slider.common.tools.CoreFileSystem.<init>(
> > > CoreFileSystem.java:82)
> > > at
> > > org.apache.slider.common.tools.SliderFileSystem.<init>(
> > > SliderFileSystem.java:38)
> > > at
> > > org.apache.slider.client.SliderClient.initHadoopBinding(
> > > SliderClient.java:498)
> > > at
> > > org.apache.slider.client.SliderClient.serviceInit(
> SliderClient.java:318)
> > > at
> > > org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:163)
> > > ... 5 more
> > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with
> status
> > 56
> > >
> > > I have tried (with no success) to copy the maprfs jar to the slider lib
> > > path, and to set the CLASSPATH on slider-env.sh as:
> > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
> > > and
> > > export CLASSPATH=`hadoop classpath`
> > >
> > > Has anybody got apache slider to work on top of mapr?
> > >
> > > Thanks!
> > > Nicolas
> > >
> >
>
>
>
> --
> Nicolás Velásquez O.
> Mobile +41.767189348
>
--
Sent from some random computer....
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by "Nicolás Velásquez O." <gn...@gmail.com>.
Hey,
It's great to initiate MapR to slider!
Unfortunately, following the suggestions did not work (remove the classpath
modification on slider-env.sh and copied mapr's jars to slider's lib
folder).
I'm guessing support for parsing the fs.defaultFS string ("maprfs:///") has
to be added to Slider's code. Something similar to
https://issues.apache.org/jira/browse/FLINK-1006.
Should I just open an incident on Jira?
Thanks!
Nicolas
On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi <bi...@gmail.com>
wrote:
> I don't think anyone has tried running Slider on MapR yet, so you are
> breaking new ground. I would not try to change CLASSPATH in slider-env.sh.
> Slider has a standalone lib directory, meaning that all of its required
> jars are in the lib dir (including hadoop jars). It might be the case that
> more jars are needed from the mapr lib directory. I would try copying all
> the jars from the mapr lib directory to the slider lib directory and see if
> that makes a difference. If it works, then you might be able to remove some
> of the jars.
>
> On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <gn...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm having problems to run Solr using Apache Slider on MapR.
> Specifically,
> > when I try to install the slider package it fails because slider doesn't
> > have support for maprfs:
> > $ slider install-package --replacepkg --name solr --package
> > ~/inst/solr/solr-slider/solr-on-yarn.zip
> > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService - Service
> > Slider Client failed in state INITED; cause: java.io.IOException: No
> > FileSystem for scheme: maprfs
> > java.io.IOException: No FileSystem for scheme: maprfs
> > at
> > org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
> > at
> > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> > at org.apache.hadoop.fs.FileSystem.access$200(
> FileSystem.java:92)
> > at
> > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
> > at org.apache.hadoop.fs.FileSystem$Cache.get(
> FileSystem.java:2669)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> > at
> > org.apache.slider.common.tools.CoreFileSystem.<init>(
> > CoreFileSystem.java:82)
> > at
> > org.apache.slider.common.tools.SliderFileSystem.<init>(
> > SliderFileSystem.java:38)
> > at
> > org.apache.slider.client.SliderClient.initHadoopBinding(
> > SliderClient.java:498)
> > at
> > org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
> > at
> > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchService(
> > ServiceLauncher.java:182)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > ServiceLauncher.java:475)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > ServiceLauncher.java:403)
> > at
> > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > ServiceLauncher.java:630)
> > at org.apache.slider.Slider.main(Slider.java:49)
> > Exception: java.io.IOException: No FileSystem for scheme: maprfs
> > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception:
> > java.io.IOException: No FileSystem for scheme: maprfs
> > org.apache.hadoop.service.ServiceStateException: java.io.IOException: No
> > FileSystem for scheme: maprfs
> > at
> > org.apache.hadoop.service.ServiceStateException.convert(
> > ServiceStateException.java:59)
> > at
> > org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchService(
> > ServiceLauncher.java:182)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> > ServiceLauncher.java:475)
> > at
> > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> > ServiceLauncher.java:403)
> > at
> > org.apache.slider.core.main.ServiceLauncher.serviceMain(
> > ServiceLauncher.java:630)
> > at org.apache.slider.Slider.main(Slider.java:49)
> > Caused by: java.io.IOException: No FileSystem for scheme: maprfs
> > at
> > org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
> > at
> > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> > at org.apache.hadoop.fs.FileSystem.access$200(
> FileSystem.java:92)
> > at
> > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
> > at org.apache.hadoop.fs.FileSystem$Cache.get(
> FileSystem.java:2669)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> > at
> > org.apache.slider.common.tools.CoreFileSystem.<init>(
> > CoreFileSystem.java:82)
> > at
> > org.apache.slider.common.tools.SliderFileSystem.<init>(
> > SliderFileSystem.java:38)
> > at
> > org.apache.slider.client.SliderClient.initHadoopBinding(
> > SliderClient.java:498)
> > at
> > org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
> > at
> > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> > ... 5 more
> > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with status
> 56
> >
> > I have tried (with no success) to copy the maprfs jar to the slider lib
> > path, and to set the CLASSPATH on slider-env.sh as:
> > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
> > and
> > export CLASSPATH=`hadoop classpath`
> >
> > Has anybody got apache slider to work on top of mapr?
> >
> > Thanks!
> > Nicolas
> >
>
--
Nicolás Velásquez O.
Mobile +41.767189348
Re: MapR problem. No FileSystem for scheme: maprfs
Posted by Billie Rinaldi <bi...@gmail.com>.
I don't think anyone has tried running Slider on MapR yet, so you are
breaking new ground. I would not try to change CLASSPATH in slider-env.sh.
Slider has a standalone lib directory, meaning that all of its required
jars are in the lib dir (including hadoop jars). It might be the case that
more jars are needed from the mapr lib directory. I would try copying all
the jars from the mapr lib directory to the slider lib directory and see if
that makes a difference. If it works, then you might be able to remove some
of the jars.
On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. <gn...@gmail.com>
wrote:
> Hi,
>
> I'm having problems to run Solr using Apache Slider on MapR. Specifically,
> when I try to install the slider package it fails because slider doesn't
> have support for maprfs:
> $ slider install-package --replacepkg --name solr --package
> ~/inst/solr/solr-slider/solr-on-yarn.zip
> 2017-02-17 12:08:28,635 [main] INFO service.AbstractService - Service
> Slider Client failed in state INITED; cause: java.io.IOException: No
> FileSystem for scheme: maprfs
> java.io.IOException: No FileSystem for scheme: maprfs
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> at
> org.apache.slider.common.tools.CoreFileSystem.<init>(
> CoreFileSystem.java:82)
> at
> org.apache.slider.common.tools.SliderFileSystem.<init>(
> SliderFileSystem.java:38)
> at
> org.apache.slider.client.SliderClient.initHadoopBinding(
> SliderClient.java:498)
> at
> org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(
> ServiceLauncher.java:182)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> ServiceLauncher.java:475)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(
> ServiceLauncher.java:630)
> at org.apache.slider.Slider.main(Slider.java:49)
> Exception: java.io.IOException: No FileSystem for scheme: maprfs
> 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception:
> java.io.IOException: No FileSystem for scheme: maprfs
> org.apache.hadoop.service.ServiceStateException: java.io.IOException: No
> FileSystem for scheme: maprfs
> at
> org.apache.hadoop.service.ServiceStateException.convert(
> ServiceStateException.java:59)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(
> ServiceLauncher.java:182)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(
> ServiceLauncher.java:475)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(
> ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(
> ServiceLauncher.java:630)
> at org.apache.slider.Slider.main(Slider.java:49)
> Caused by: java.io.IOException: No FileSystem for scheme: maprfs
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
> at
> org.apache.slider.common.tools.CoreFileSystem.<init>(
> CoreFileSystem.java:82)
> at
> org.apache.slider.common.tools.SliderFileSystem.<init>(
> SliderFileSystem.java:38)
> at
> org.apache.slider.client.SliderClient.initHadoopBinding(
> SliderClient.java:498)
> at
> org.apache.slider.client.SliderClient.serviceInit(SliderClient.java:318)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> ... 5 more
> 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with status 56
>
> I have tried (with no success) to copy the maprfs jar to the slider lib
> path, and to set the CLASSPATH on slider-env.sh as:
> export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar
> and
> export CLASSPATH=`hadoop classpath`
>
> Has anybody got apache slider to work on top of mapr?
>
> Thanks!
> Nicolas
>