You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Ivan Veselovsky <iv...@gridgain.com> on 2015/11/17 12:19:24 UTC
Re: IllegalArgumentException: Invalid URI for NameNode address
(check fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Hi, Robert,
/please properly subscribe to the user list (this way we will not have to
manually approve your emails). All you need to do is send an email to
“user-subscribe@ignite.apache.org” and follow simple instructions in the
reply.
/
The problem is that "setup-hadoop.sh" changes global Hadoop configs, and
after that all Hadoop services, including namenode and datanode are using
it. But Hadoop namenode and datanode cannot work over IGFS, namenode
explicitly checks that -- this is the cause of the error you see.
Relevant doc page: https://apacheignite.readme.io/docs/file-system says:
-----
Second, you may override default core-site.xml of your Hadoop installation.
This will force all Hadoop jobs to pick Ignite jobs tracker by default
unless it is overriden on job level somehow. Note that you will not be able
to HDFS in this case.
-----
If you change the global configs as "setup-hadoop.sh" does, you will not be
able to use HDFS services any more, and will be able to run IGFS in
"primary" mode only (see https://apacheignite.readme.io/docs/modes).
Way to fix the situation is described there:
https://issues.apache.org/jira/browse/IGNITE-483 .
"ls: Failed to communicate with IGFS." is another problem that indicates ,
that Hadoop client for some reason fails to connect to IGFS node. First of
all , is the Ignite node started correctly? If the problem persists, can you
please attach configs and logs.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p1985.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IllegalArgumentException: Invalid URI for NameNode address
(check fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Posted by RobertOfFairfax <rg...@gmail.com>.
Hi Denis,
All is well.
Thanks,
Robert
On Fri, Nov 20, 2015 at 5:21 AM, Denis Magda [via Apache Ignite Users] <
ml-node+s70518n2021h84@n6.nabble.com> wrote:
> Hi Robert,
>
> So, no more issues so far with this and workarounds work fine for you,
> right?
>
> --
> Denis
>
> On 11/19/2015 5:43 PM, RobertOfFairfax wrote:
>
> > Hi Ivan,
> >
> > The hadoop wrapper example in IGNITE-483
> > <https://issues.apache.org/jira/browse/IGNITE-483> with an updated
> > HADOOP_CLASSPATH worked great as well as understanding that you can't
> get
> > this working until Ignite is running.
> >
> > In summary, I
> >
> > 1) copied the core-site-ignite.xml and mapred-site-ignite.xml files to
> > $IGNITE_HOME/ignite-conf/
> >
> > 2) renamed them to core-site-ignite.xml and mapred-site-ignite.xml
> >
> > 3) created the hadoop-ignite wrapper script similar to that described in
> > IGNITE-483
> >
> > 4) started Ignite with '$IGNITE_HOME/bin/ignite.sh
> > $IGNITE_HOME/config/default-config.xml'
> >
> > Further details of it's usage are described here
> > <https://github.com/rkennington/ignite-docker> .
> >
> > Thanks,
> > Robert
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2017.html
> > Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2021.html
> To unsubscribe from IllegalArgumentException: Invalid URI for NameNode
> address (check fs.defaultFS): igfs://igfs@localhost is not of scheme
> 'hdfs', click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1978&code=cmdrLmJpekBnbWFpbC5jb218MTk3OHw3Mzg4NjA5Ng==>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2029.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IllegalArgumentException: Invalid URI for NameNode address (check
fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Posted by Denis Magda <dm...@gridgain.com>.
Hi Robert,
So, no more issues so far with this and workarounds work fine for you,
right?
--
Denis
On 11/19/2015 5:43 PM, RobertOfFairfax wrote:
> Hi Ivan,
>
> The hadoop wrapper example in IGNITE-483
> <https://issues.apache.org/jira/browse/IGNITE-483> with an updated
> HADOOP_CLASSPATH worked great as well as understanding that you can't get
> this working until Ignite is running.
>
> In summary, I
>
> 1) copied the core-site-ignite.xml and mapred-site-ignite.xml files to
> $IGNITE_HOME/ignite-conf/
>
> 2) renamed them to core-site-ignite.xml and mapred-site-ignite.xml
>
> 3) created the hadoop-ignite wrapper script similar to that described in
> IGNITE-483
>
> 4) started Ignite with '$IGNITE_HOME/bin/ignite.sh
> $IGNITE_HOME/config/default-config.xml'
>
> Further details of it's usage are described here
> <https://github.com/rkennington/ignite-docker> .
>
> Thanks,
> Robert
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2017.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IllegalArgumentException: Invalid URI for NameNode address
(check fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Posted by RobertOfFairfax <rg...@gmail.com>.
Hi Ivan,
The hadoop wrapper example in IGNITE-483
<https://issues.apache.org/jira/browse/IGNITE-483> with an updated
HADOOP_CLASSPATH worked great as well as understanding that you can't get
this working until Ignite is running.
In summary, I
1) copied the core-site-ignite.xml and mapred-site-ignite.xml files to
$IGNITE_HOME/ignite-conf/
2) renamed them to core-site-ignite.xml and mapred-site-ignite.xml
3) created the hadoop-ignite wrapper script similar to that described in
IGNITE-483
4) started Ignite with '$IGNITE_HOME/bin/ignite.sh
$IGNITE_HOME/config/default-config.xml'
Further details of it's usage are described here
<https://github.com/rkennington/ignite-docker> .
Thanks,
Robert
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2017.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IllegalArgumentException: Invalid URI for NameNode address
(check fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Posted by RobertOfFairfax <rg...@gmail.com>.
Hi Ivan, Denis,
Thanks for your prompt response. Currently, my goal is to observe
performance with and without IGFS using a stream of Pig commands against
delimited files.
> Did you have a chance to apply the steps described in IGNITE-483?
This is a weekend activity with the following objectives for which I have
described in a docker project (https://github.com/rkennington/ignite-docker
):
1) Configure Hadoop to integrate with IGFS based on installing it on
cloudera CDH
<https://apacheignite.readme.io/docs/installing-on-cloudera-cdh>. - as well
as confirm it is working with a few CLI commands. Load some comma delimited
files to work with.
2) Startup one or more Ignite clients to use it as a resource.
3) Develop a client that distributes and executes a stream of Pig commands.
Continue to use the Docker platform as a development environment to
experiment with.
4) Migrate it to a distributed cluster to observe performance.
5) Explore other aspects of Ignite use cases.
> However probably you're planning to use IGFS is some other mode and you
don't need to fix any there.
You may be right. I could be over thinking this problem with old hadoop
file system CLI biases for which I may need to unlearn. As I move along,
I'm trying to understand what CLI commands I can use to verify my setup.
Maybe the
'hadoop fs -ls /'
command is not of value in this context.
> If something is unclear to you let us know and clarify how you're going
to use IGFS (in which mode).
Will do, thanks.
Robert
571-882-2580 m
On Wed, Nov 18, 2015 at 4:57 AM, Denis Magda [via Apache Ignite Users] <
ml-node+s70518n2005h88@n6.nabble.com> wrote:
> Robert,
>
> Did you have a chance to apply the steps described in IGNITE-483?
> However probably you're planning to use IGFS is some other mode and you
> don't need to fix any there.
>
> If something is unclear to you let us know and clarify how you're going to
> use IGFS (in which mode).
>
>
> --
> Denis
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2005.html
> To unsubscribe from IllegalArgumentException: Invalid URI for NameNode
> address (check fs.defaultFS): igfs://igfs@localhost is not of scheme
> 'hdfs', click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1978&code=cmdrLmJpekBnbWFpbC5jb218MTk3OHw3Mzg4NjA5Ng==>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2009.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IllegalArgumentException: Invalid URI for NameNode address
(check fs.defaultFS): igfs://igfs@localhost is not of scheme 'hdfs'
Posted by Denis Magda <dm...@gridgain.com>.
Robert,
Did you have a chance to apply the steps described in IGNITE-483?
However probably you're planning to use IGFS is some other mode and you
don't need to fix any there.
If something is unclear to you let us know and clarify how you're going to
use IGFS (in which mode).
--
Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IllegalArgumentException-Invalid-URI-for-NameNode-address-check-fs-defaultFS-igfs-igfs-localhost-is--tp1978p2005.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.