You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Alexander Zarei <al...@gmail.com> on 2015/04/03 05:51:35 UTC
Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Hi,
I am suing Drill on an AWS EMR cluster and trying to connect Hive and
MapR-FS storage plugins.
The hive plugin is connected successfully as inlined below; I created a
dummy table by hive shell and then I observed the following Drill Error
when browsing the dummy table with Drill Exporer:
*[30024]Query execution error. Details:[ IOException: No FileSystem for
scheme: maprfs*
I tried to connect the MapR-FS storage plugin but I am not sure if the port
number I am using is correct.
According to Drill documentation any distributed file system included in
core-site.xml can be used by Drill but I do not see any information related
to MapR-FS in this file. However, when I use the Amazon Hadoop the
information (hdfs nodename ip and port number ) is there and I can connect
to hive storage by Drill.
I wonder if you could let me know what I am missing in crafting the storage
plugins below:
Hive storage plugin:
{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "thrift://localhost:9083",
"hive.metastore.local": "false",
"hive.metastore.warehouse.dir": "/user/hive/warehouse"
}
}
and maprfs:
{
"type": "file",
"enabled": "false",
"connection": "maprfs://54.224.109.121:8041/",
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null
}
},
"formats": {
"psv": {
"type": "text",
"extensions": [
"tbl"
],
"delimiter": "|"
},
"csv": {
"type": "text",
"extensions": [
"csv"
],
"delimiter": ","
},
"tsv": {
"type": "text",
"extensions": [
"tsv"
],
"delimiter": "\t"
},
"parquet": {
"type": "parquet"
},
"json": {
"type": "json"
}
}
}
Thanks,
Alex
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Alexander Zarei <al...@gmail.com>.
Thanks a lot!! I will build and try it with MapR profile!
Thanks,
Alex
On Thu, Apr 16, 2015 at 1:04 PM, Steven Phillips <sp...@maprtech.com>
wrote:
> I believe getdrill.org points to the official Apache releases, which are
> not built with mapr profile.
>
> On Thu, Apr 16, 2015 at 12:54 PM, Alexander Zarei <
> alexanderz.simba@gmail.com> wrote:
>
> > That is actually a good point. Thanks for mentioning it.
> >
> > It is the one grabbed from here:
> > http://getdrill.org/drill/download/apache-drill-0.8.0.tar.gz
> >
> > I am actually not sure if it is compile with -Pmapr profile. Do you guys
> > know if the pre built ones are built with the -Pmapr profile?
> >
> > Thanks,
> > Alex
> >
> > On Wed, Apr 15, 2015 at 11:19 AM, Venki Korukanti <
> > venki.korukanti@gmail.com
> > > wrote:
> >
> > > Is the tar.gz file generated compiled with -Pmapr profle?
> > >
> > > On Wed, Apr 15, 2015 at 11:11 AM, Alexander Zarei <
> > > alexanderz.simba@gmail.com> wrote:
> > >
> > > > Thanks Ramana for the help. I tried what you said but it did not
> work.
> > > > Basically the storage plugin addition fails. I set the log level to
> > debug
> > > > and it didn't tell much about the problem. I will try it with TRACE
> log
> > > > level to see what happened.
> > > >
> > > > As I was not able to get the MapR Hadoop on EMR running I switched to
> > > > Amazon Hadoop and it worked as per documentation.
> > > >
> > > > Thanks again for your reply.
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > > > On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <
> > rinukonda@maprtech.com>
> > > > wrote:
> > > >
> > > > > Also is drill running on the same node as a MapR cluster? In that
> > case
> > > > you
> > > > > just can specify maprfs:/// for the connection value.
> > > > >
> > > > > Regards
> > > > > Ramana
> > > > >
> > > > >
> > > > > On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <
> > > rinukonda@maprtech.com>
> > > > > wrote:
> > > > >
> > > > > > Please check if your drill-env.sh contains the following:
> > > > > >
> > > > > > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > > > > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > > > > -Dzookeeper.sasl.client=false "
> > > > > >
> > > > > >
> > > > > > Regards
> > > > > > Ramana
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > > > > > alexanderz.simba@gmail.com> wrote:
> > > > > >
> > > > > >> Hi,
> > > > > >>
> > > > > >>
> > > > > >> I am suing Drill on an AWS EMR cluster and trying to connect
> Hive
> > > and
> > > > > >> MapR-FS storage plugins.
> > > > > >>
> > > > > >> The hive plugin is connected successfully as inlined below; I
> > > created
> > > > a
> > > > > >> dummy table by hive shell and then I observed the following
> Drill
> > > > Error
> > > > > >> when browsing the dummy table with Drill Exporer:
> > > > > >>
> > > > > >>
> > > > > >> *[30024]Query execution error. Details:[ IOException: No
> > FileSystem
> > > > for
> > > > > >> scheme: maprfs*
> > > > > >>
> > > > > >> I tried to connect the MapR-FS storage plugin but I am not sure
> if
> > > the
> > > > > >> port
> > > > > >> number I am using is correct.
> > > > > >>
> > > > > >> According to Drill documentation any distributed file system
> > > included
> > > > in
> > > > > >> core-site.xml can be used by Drill but I do not see any
> > information
> > > > > >> related
> > > > > >> to MapR-FS in this file. However, when I use the Amazon Hadoop
> the
> > > > > >> information (hdfs nodename ip and port number ) is there and I
> can
> > > > > connect
> > > > > >> to hive storage by Drill.
> > > > > >>
> > > > > >> I wonder if you could let me know what I am missing in crafting
> > the
> > > > > >> storage
> > > > > >> plugins below:
> > > > > >>
> > > > > >> Hive storage plugin:
> > > > > >>
> > > > > >> {
> > > > > >> "type": "hive",
> > > > > >> "enabled": true,
> > > > > >> "configProps": {
> > > > > >> "hive.metastore.uris": "thrift://localhost:9083",
> > > > > >> "hive.metastore.local": "false",
> > > > > >> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> > > > > >> }
> > > > > >> }
> > > > > >>
> > > > > >> and maprfs:
> > > > > >>
> > > > > >> {
> > > > > >> "type": "file",
> > > > > >> "enabled": "false",
> > > > > >> "connection": "maprfs://54.224.109.121:8041/",
> > > > > >> "workspaces": {
> > > > > >> "root": {
> > > > > >> "location": "/",
> > > > > >> "writable": false,
> > > > > >> "defaultInputFormat": null
> > > > > >> },
> > > > > >> "tmp": {
> > > > > >> "location": "/tmp",
> > > > > >> "writable": true,
> > > > > >> "defaultInputFormat": null
> > > > > >> }
> > > > > >> },
> > > > > >> "formats": {
> > > > > >> "psv": {
> > > > > >> "type": "text",
> > > > > >> "extensions": [
> > > > > >> "tbl"
> > > > > >> ],
> > > > > >> "delimiter": "|"
> > > > > >> },
> > > > > >> "csv": {
> > > > > >> "type": "text",
> > > > > >> "extensions": [
> > > > > >> "csv"
> > > > > >> ],
> > > > > >> "delimiter": ","
> > > > > >> },
> > > > > >> "tsv": {
> > > > > >> "type": "text",
> > > > > >> "extensions": [
> > > > > >> "tsv"
> > > > > >> ],
> > > > > >> "delimiter": "\t"
> > > > > >> },
> > > > > >> "parquet": {
> > > > > >> "type": "parquet"
> > > > > >> },
> > > > > >> "json": {
> > > > > >> "type": "json"
> > > > > >> }
> > > > > >> }
> > > > > >> }
> > > > > >>
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Alex
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Steven Phillips
> Software Engineer
>
> mapr.com
>
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Steven Phillips <sp...@maprtech.com>.
I believe getdrill.org points to the official Apache releases, which are
not built with mapr profile.
On Thu, Apr 16, 2015 at 12:54 PM, Alexander Zarei <
alexanderz.simba@gmail.com> wrote:
> That is actually a good point. Thanks for mentioning it.
>
> It is the one grabbed from here:
> http://getdrill.org/drill/download/apache-drill-0.8.0.tar.gz
>
> I am actually not sure if it is compile with -Pmapr profile. Do you guys
> know if the pre built ones are built with the -Pmapr profile?
>
> Thanks,
> Alex
>
> On Wed, Apr 15, 2015 at 11:19 AM, Venki Korukanti <
> venki.korukanti@gmail.com
> > wrote:
>
> > Is the tar.gz file generated compiled with -Pmapr profle?
> >
> > On Wed, Apr 15, 2015 at 11:11 AM, Alexander Zarei <
> > alexanderz.simba@gmail.com> wrote:
> >
> > > Thanks Ramana for the help. I tried what you said but it did not work.
> > > Basically the storage plugin addition fails. I set the log level to
> debug
> > > and it didn't tell much about the problem. I will try it with TRACE log
> > > level to see what happened.
> > >
> > > As I was not able to get the MapR Hadoop on EMR running I switched to
> > > Amazon Hadoop and it worked as per documentation.
> > >
> > > Thanks again for your reply.
> > >
> > > Thanks,
> > > Alex
> > >
> > > On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <
> rinukonda@maprtech.com>
> > > wrote:
> > >
> > > > Also is drill running on the same node as a MapR cluster? In that
> case
> > > you
> > > > just can specify maprfs:/// for the connection value.
> > > >
> > > > Regards
> > > > Ramana
> > > >
> > > >
> > > > On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <
> > rinukonda@maprtech.com>
> > > > wrote:
> > > >
> > > > > Please check if your drill-env.sh contains the following:
> > > > >
> > > > > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > > > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > > > -Dzookeeper.sasl.client=false "
> > > > >
> > > > >
> > > > > Regards
> > > > > Ramana
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > > > > alexanderz.simba@gmail.com> wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >>
> > > > >> I am suing Drill on an AWS EMR cluster and trying to connect Hive
> > and
> > > > >> MapR-FS storage plugins.
> > > > >>
> > > > >> The hive plugin is connected successfully as inlined below; I
> > created
> > > a
> > > > >> dummy table by hive shell and then I observed the following Drill
> > > Error
> > > > >> when browsing the dummy table with Drill Exporer:
> > > > >>
> > > > >>
> > > > >> *[30024]Query execution error. Details:[ IOException: No
> FileSystem
> > > for
> > > > >> scheme: maprfs*
> > > > >>
> > > > >> I tried to connect the MapR-FS storage plugin but I am not sure if
> > the
> > > > >> port
> > > > >> number I am using is correct.
> > > > >>
> > > > >> According to Drill documentation any distributed file system
> > included
> > > in
> > > > >> core-site.xml can be used by Drill but I do not see any
> information
> > > > >> related
> > > > >> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> > > > >> information (hdfs nodename ip and port number ) is there and I can
> > > > connect
> > > > >> to hive storage by Drill.
> > > > >>
> > > > >> I wonder if you could let me know what I am missing in crafting
> the
> > > > >> storage
> > > > >> plugins below:
> > > > >>
> > > > >> Hive storage plugin:
> > > > >>
> > > > >> {
> > > > >> "type": "hive",
> > > > >> "enabled": true,
> > > > >> "configProps": {
> > > > >> "hive.metastore.uris": "thrift://localhost:9083",
> > > > >> "hive.metastore.local": "false",
> > > > >> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> > > > >> }
> > > > >> }
> > > > >>
> > > > >> and maprfs:
> > > > >>
> > > > >> {
> > > > >> "type": "file",
> > > > >> "enabled": "false",
> > > > >> "connection": "maprfs://54.224.109.121:8041/",
> > > > >> "workspaces": {
> > > > >> "root": {
> > > > >> "location": "/",
> > > > >> "writable": false,
> > > > >> "defaultInputFormat": null
> > > > >> },
> > > > >> "tmp": {
> > > > >> "location": "/tmp",
> > > > >> "writable": true,
> > > > >> "defaultInputFormat": null
> > > > >> }
> > > > >> },
> > > > >> "formats": {
> > > > >> "psv": {
> > > > >> "type": "text",
> > > > >> "extensions": [
> > > > >> "tbl"
> > > > >> ],
> > > > >> "delimiter": "|"
> > > > >> },
> > > > >> "csv": {
> > > > >> "type": "text",
> > > > >> "extensions": [
> > > > >> "csv"
> > > > >> ],
> > > > >> "delimiter": ","
> > > > >> },
> > > > >> "tsv": {
> > > > >> "type": "text",
> > > > >> "extensions": [
> > > > >> "tsv"
> > > > >> ],
> > > > >> "delimiter": "\t"
> > > > >> },
> > > > >> "parquet": {
> > > > >> "type": "parquet"
> > > > >> },
> > > > >> "json": {
> > > > >> "type": "json"
> > > > >> }
> > > > >> }
> > > > >> }
> > > > >>
> > > > >>
> > > > >> Thanks,
> > > > >> Alex
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Alexander Zarei <al...@gmail.com>.
That is actually a good point. Thanks for mentioning it.
It is the one grabbed from here:
http://getdrill.org/drill/download/apache-drill-0.8.0.tar.gz
I am actually not sure if it is compile with -Pmapr profile. Do you guys
know if the pre built ones are built with the -Pmapr profile?
Thanks,
Alex
On Wed, Apr 15, 2015 at 11:19 AM, Venki Korukanti <venki.korukanti@gmail.com
> wrote:
> Is the tar.gz file generated compiled with -Pmapr profle?
>
> On Wed, Apr 15, 2015 at 11:11 AM, Alexander Zarei <
> alexanderz.simba@gmail.com> wrote:
>
> > Thanks Ramana for the help. I tried what you said but it did not work.
> > Basically the storage plugin addition fails. I set the log level to debug
> > and it didn't tell much about the problem. I will try it with TRACE log
> > level to see what happened.
> >
> > As I was not able to get the MapR Hadoop on EMR running I switched to
> > Amazon Hadoop and it worked as per documentation.
> >
> > Thanks again for your reply.
> >
> > Thanks,
> > Alex
> >
> > On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <ri...@maprtech.com>
> > wrote:
> >
> > > Also is drill running on the same node as a MapR cluster? In that case
> > you
> > > just can specify maprfs:/// for the connection value.
> > >
> > > Regards
> > > Ramana
> > >
> > >
> > > On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <
> rinukonda@maprtech.com>
> > > wrote:
> > >
> > > > Please check if your drill-env.sh contains the following:
> > > >
> > > > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > > -Dzookeeper.sasl.client=false "
> > > >
> > > >
> > > > Regards
> > > > Ramana
> > > >
> > > >
> > > >
> > > > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > > > alexanderz.simba@gmail.com> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>
> > > >> I am suing Drill on an AWS EMR cluster and trying to connect Hive
> and
> > > >> MapR-FS storage plugins.
> > > >>
> > > >> The hive plugin is connected successfully as inlined below; I
> created
> > a
> > > >> dummy table by hive shell and then I observed the following Drill
> > Error
> > > >> when browsing the dummy table with Drill Exporer:
> > > >>
> > > >>
> > > >> *[30024]Query execution error. Details:[ IOException: No FileSystem
> > for
> > > >> scheme: maprfs*
> > > >>
> > > >> I tried to connect the MapR-FS storage plugin but I am not sure if
> the
> > > >> port
> > > >> number I am using is correct.
> > > >>
> > > >> According to Drill documentation any distributed file system
> included
> > in
> > > >> core-site.xml can be used by Drill but I do not see any information
> > > >> related
> > > >> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> > > >> information (hdfs nodename ip and port number ) is there and I can
> > > connect
> > > >> to hive storage by Drill.
> > > >>
> > > >> I wonder if you could let me know what I am missing in crafting the
> > > >> storage
> > > >> plugins below:
> > > >>
> > > >> Hive storage plugin:
> > > >>
> > > >> {
> > > >> "type": "hive",
> > > >> "enabled": true,
> > > >> "configProps": {
> > > >> "hive.metastore.uris": "thrift://localhost:9083",
> > > >> "hive.metastore.local": "false",
> > > >> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> > > >> }
> > > >> }
> > > >>
> > > >> and maprfs:
> > > >>
> > > >> {
> > > >> "type": "file",
> > > >> "enabled": "false",
> > > >> "connection": "maprfs://54.224.109.121:8041/",
> > > >> "workspaces": {
> > > >> "root": {
> > > >> "location": "/",
> > > >> "writable": false,
> > > >> "defaultInputFormat": null
> > > >> },
> > > >> "tmp": {
> > > >> "location": "/tmp",
> > > >> "writable": true,
> > > >> "defaultInputFormat": null
> > > >> }
> > > >> },
> > > >> "formats": {
> > > >> "psv": {
> > > >> "type": "text",
> > > >> "extensions": [
> > > >> "tbl"
> > > >> ],
> > > >> "delimiter": "|"
> > > >> },
> > > >> "csv": {
> > > >> "type": "text",
> > > >> "extensions": [
> > > >> "csv"
> > > >> ],
> > > >> "delimiter": ","
> > > >> },
> > > >> "tsv": {
> > > >> "type": "text",
> > > >> "extensions": [
> > > >> "tsv"
> > > >> ],
> > > >> "delimiter": "\t"
> > > >> },
> > > >> "parquet": {
> > > >> "type": "parquet"
> > > >> },
> > > >> "json": {
> > > >> "type": "json"
> > > >> }
> > > >> }
> > > >> }
> > > >>
> > > >>
> > > >> Thanks,
> > > >> Alex
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Venki Korukanti <ve...@gmail.com>.
Is the tar.gz file generated compiled with -Pmapr profle?
On Wed, Apr 15, 2015 at 11:11 AM, Alexander Zarei <
alexanderz.simba@gmail.com> wrote:
> Thanks Ramana for the help. I tried what you said but it did not work.
> Basically the storage plugin addition fails. I set the log level to debug
> and it didn't tell much about the problem. I will try it with TRACE log
> level to see what happened.
>
> As I was not able to get the MapR Hadoop on EMR running I switched to
> Amazon Hadoop and it worked as per documentation.
>
> Thanks again for your reply.
>
> Thanks,
> Alex
>
> On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <ri...@maprtech.com>
> wrote:
>
> > Also is drill running on the same node as a MapR cluster? In that case
> you
> > just can specify maprfs:/// for the connection value.
> >
> > Regards
> > Ramana
> >
> >
> > On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <ri...@maprtech.com>
> > wrote:
> >
> > > Please check if your drill-env.sh contains the following:
> > >
> > > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > -Dzookeeper.sasl.client=false "
> > >
> > >
> > > Regards
> > > Ramana
> > >
> > >
> > >
> > > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > > alexanderz.simba@gmail.com> wrote:
> > >
> > >> Hi,
> > >>
> > >>
> > >> I am suing Drill on an AWS EMR cluster and trying to connect Hive and
> > >> MapR-FS storage plugins.
> > >>
> > >> The hive plugin is connected successfully as inlined below; I created
> a
> > >> dummy table by hive shell and then I observed the following Drill
> Error
> > >> when browsing the dummy table with Drill Exporer:
> > >>
> > >>
> > >> *[30024]Query execution error. Details:[ IOException: No FileSystem
> for
> > >> scheme: maprfs*
> > >>
> > >> I tried to connect the MapR-FS storage plugin but I am not sure if the
> > >> port
> > >> number I am using is correct.
> > >>
> > >> According to Drill documentation any distributed file system included
> in
> > >> core-site.xml can be used by Drill but I do not see any information
> > >> related
> > >> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> > >> information (hdfs nodename ip and port number ) is there and I can
> > connect
> > >> to hive storage by Drill.
> > >>
> > >> I wonder if you could let me know what I am missing in crafting the
> > >> storage
> > >> plugins below:
> > >>
> > >> Hive storage plugin:
> > >>
> > >> {
> > >> "type": "hive",
> > >> "enabled": true,
> > >> "configProps": {
> > >> "hive.metastore.uris": "thrift://localhost:9083",
> > >> "hive.metastore.local": "false",
> > >> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> > >> }
> > >> }
> > >>
> > >> and maprfs:
> > >>
> > >> {
> > >> "type": "file",
> > >> "enabled": "false",
> > >> "connection": "maprfs://54.224.109.121:8041/",
> > >> "workspaces": {
> > >> "root": {
> > >> "location": "/",
> > >> "writable": false,
> > >> "defaultInputFormat": null
> > >> },
> > >> "tmp": {
> > >> "location": "/tmp",
> > >> "writable": true,
> > >> "defaultInputFormat": null
> > >> }
> > >> },
> > >> "formats": {
> > >> "psv": {
> > >> "type": "text",
> > >> "extensions": [
> > >> "tbl"
> > >> ],
> > >> "delimiter": "|"
> > >> },
> > >> "csv": {
> > >> "type": "text",
> > >> "extensions": [
> > >> "csv"
> > >> ],
> > >> "delimiter": ","
> > >> },
> > >> "tsv": {
> > >> "type": "text",
> > >> "extensions": [
> > >> "tsv"
> > >> ],
> > >> "delimiter": "\t"
> > >> },
> > >> "parquet": {
> > >> "type": "parquet"
> > >> },
> > >> "json": {
> > >> "type": "json"
> > >> }
> > >> }
> > >> }
> > >>
> > >>
> > >> Thanks,
> > >> Alex
> > >>
> > >
> > >
> >
>
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Alexander Zarei <al...@gmail.com>.
Thanks Ramana for the help. I tried what you said but it did not work.
Basically the storage plugin addition fails. I set the log level to debug
and it didn't tell much about the problem. I will try it with TRACE log
level to see what happened.
As I was not able to get the MapR Hadoop on EMR running I switched to
Amazon Hadoop and it worked as per documentation.
Thanks again for your reply.
Thanks,
Alex
On Fri, Apr 3, 2015 at 1:58 PM, Ramana Inukonda <ri...@maprtech.com>
wrote:
> Also is drill running on the same node as a MapR cluster? In that case you
> just can specify maprfs:/// for the connection value.
>
> Regards
> Ramana
>
>
> On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <ri...@maprtech.com>
> wrote:
>
> > Please check if your drill-env.sh contains the following:
> >
> > export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > -Dzookeeper.sasl.client=false "
> >
> >
> > Regards
> > Ramana
> >
> >
> >
> > On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> > alexanderz.simba@gmail.com> wrote:
> >
> >> Hi,
> >>
> >>
> >> I am suing Drill on an AWS EMR cluster and trying to connect Hive and
> >> MapR-FS storage plugins.
> >>
> >> The hive plugin is connected successfully as inlined below; I created a
> >> dummy table by hive shell and then I observed the following Drill Error
> >> when browsing the dummy table with Drill Exporer:
> >>
> >>
> >> *[30024]Query execution error. Details:[ IOException: No FileSystem for
> >> scheme: maprfs*
> >>
> >> I tried to connect the MapR-FS storage plugin but I am not sure if the
> >> port
> >> number I am using is correct.
> >>
> >> According to Drill documentation any distributed file system included in
> >> core-site.xml can be used by Drill but I do not see any information
> >> related
> >> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> >> information (hdfs nodename ip and port number ) is there and I can
> connect
> >> to hive storage by Drill.
> >>
> >> I wonder if you could let me know what I am missing in crafting the
> >> storage
> >> plugins below:
> >>
> >> Hive storage plugin:
> >>
> >> {
> >> "type": "hive",
> >> "enabled": true,
> >> "configProps": {
> >> "hive.metastore.uris": "thrift://localhost:9083",
> >> "hive.metastore.local": "false",
> >> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> >> }
> >> }
> >>
> >> and maprfs:
> >>
> >> {
> >> "type": "file",
> >> "enabled": "false",
> >> "connection": "maprfs://54.224.109.121:8041/",
> >> "workspaces": {
> >> "root": {
> >> "location": "/",
> >> "writable": false,
> >> "defaultInputFormat": null
> >> },
> >> "tmp": {
> >> "location": "/tmp",
> >> "writable": true,
> >> "defaultInputFormat": null
> >> }
> >> },
> >> "formats": {
> >> "psv": {
> >> "type": "text",
> >> "extensions": [
> >> "tbl"
> >> ],
> >> "delimiter": "|"
> >> },
> >> "csv": {
> >> "type": "text",
> >> "extensions": [
> >> "csv"
> >> ],
> >> "delimiter": ","
> >> },
> >> "tsv": {
> >> "type": "text",
> >> "extensions": [
> >> "tsv"
> >> ],
> >> "delimiter": "\t"
> >> },
> >> "parquet": {
> >> "type": "parquet"
> >> },
> >> "json": {
> >> "type": "json"
> >> }
> >> }
> >> }
> >>
> >>
> >> Thanks,
> >> Alex
> >>
> >
> >
>
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Ramana Inukonda <ri...@maprtech.com>.
Also is drill running on the same node as a MapR cluster? In that case you
just can specify maprfs:/// for the connection value.
Regards
Ramana
On Fri, Apr 3, 2015 at 1:40 PM, Ramana Inukonda <ri...@maprtech.com>
wrote:
> Please check if your drill-env.sh contains the following:
>
> export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
> -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> -Dzookeeper.sasl.client=false "
>
>
> Regards
> Ramana
>
>
>
> On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <
> alexanderz.simba@gmail.com> wrote:
>
>> Hi,
>>
>>
>> I am suing Drill on an AWS EMR cluster and trying to connect Hive and
>> MapR-FS storage plugins.
>>
>> The hive plugin is connected successfully as inlined below; I created a
>> dummy table by hive shell and then I observed the following Drill Error
>> when browsing the dummy table with Drill Exporer:
>>
>>
>> *[30024]Query execution error. Details:[ IOException: No FileSystem for
>> scheme: maprfs*
>>
>> I tried to connect the MapR-FS storage plugin but I am not sure if the
>> port
>> number I am using is correct.
>>
>> According to Drill documentation any distributed file system included in
>> core-site.xml can be used by Drill but I do not see any information
>> related
>> to MapR-FS in this file. However, when I use the Amazon Hadoop the
>> information (hdfs nodename ip and port number ) is there and I can connect
>> to hive storage by Drill.
>>
>> I wonder if you could let me know what I am missing in crafting the
>> storage
>> plugins below:
>>
>> Hive storage plugin:
>>
>> {
>> "type": "hive",
>> "enabled": true,
>> "configProps": {
>> "hive.metastore.uris": "thrift://localhost:9083",
>> "hive.metastore.local": "false",
>> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
>> }
>> }
>>
>> and maprfs:
>>
>> {
>> "type": "file",
>> "enabled": "false",
>> "connection": "maprfs://54.224.109.121:8041/",
>> "workspaces": {
>> "root": {
>> "location": "/",
>> "writable": false,
>> "defaultInputFormat": null
>> },
>> "tmp": {
>> "location": "/tmp",
>> "writable": true,
>> "defaultInputFormat": null
>> }
>> },
>> "formats": {
>> "psv": {
>> "type": "text",
>> "extensions": [
>> "tbl"
>> ],
>> "delimiter": "|"
>> },
>> "csv": {
>> "type": "text",
>> "extensions": [
>> "csv"
>> ],
>> "delimiter": ","
>> },
>> "tsv": {
>> "type": "text",
>> "extensions": [
>> "tsv"
>> ],
>> "delimiter": "\t"
>> },
>> "parquet": {
>> "type": "parquet"
>> },
>> "json": {
>> "type": "json"
>> }
>> }
>> }
>>
>>
>> Thanks,
>> Alex
>>
>
>
Re: Error using Hive storage Plugin: IOException: No FileSystem for
scheme: maprfs
Posted by Ramana Inukonda <ri...@maprtech.com>.
Please check if your drill-env.sh contains the following:
export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS
-Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
-Dzookeeper.sasl.client=false "
Regards
Ramana
On Thu, Apr 2, 2015 at 8:51 PM, Alexander Zarei <al...@gmail.com>
wrote:
> Hi,
>
>
> I am suing Drill on an AWS EMR cluster and trying to connect Hive and
> MapR-FS storage plugins.
>
> The hive plugin is connected successfully as inlined below; I created a
> dummy table by hive shell and then I observed the following Drill Error
> when browsing the dummy table with Drill Exporer:
>
>
> *[30024]Query execution error. Details:[ IOException: No FileSystem for
> scheme: maprfs*
>
> I tried to connect the MapR-FS storage plugin but I am not sure if the port
> number I am using is correct.
>
> According to Drill documentation any distributed file system included in
> core-site.xml can be used by Drill but I do not see any information related
> to MapR-FS in this file. However, when I use the Amazon Hadoop the
> information (hdfs nodename ip and port number ) is there and I can connect
> to hive storage by Drill.
>
> I wonder if you could let me know what I am missing in crafting the storage
> plugins below:
>
> Hive storage plugin:
>
> {
> "type": "hive",
> "enabled": true,
> "configProps": {
> "hive.metastore.uris": "thrift://localhost:9083",
> "hive.metastore.local": "false",
> "hive.metastore.warehouse.dir": "/user/hive/warehouse"
> }
> }
>
> and maprfs:
>
> {
> "type": "file",
> "enabled": "false",
> "connection": "maprfs://54.224.109.121:8041/",
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "defaultInputFormat": null
> },
> "tmp": {
> "location": "/tmp",
> "writable": true,
> "defaultInputFormat": null
> }
> },
> "formats": {
> "psv": {
> "type": "text",
> "extensions": [
> "tbl"
> ],
> "delimiter": "|"
> },
> "csv": {
> "type": "text",
> "extensions": [
> "csv"
> ],
> "delimiter": ","
> },
> "tsv": {
> "type": "text",
> "extensions": [
> "tsv"
> ],
> "delimiter": "\t"
> },
> "parquet": {
> "type": "parquet"
> },
> "json": {
> "type": "json"
> }
> }
> }
>
>
> Thanks,
> Alex
>