You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com> on 2012/08/09 09:07:44 UTC
Question on adding Hadoop XML to Configuration Object
Hi,
I am trying to add a file to HDFS programmatically.
In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration object as follows
Configuration configuration = null;
configuration.addResource(new URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
configuration.addResource(new Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
configuration.reloadConfiguration();
fileSystem = FileSystem.get(configuration);
System.out.println(fileSystem.getName());
This code prints file system as file:/// instead of hdfs://.
When I manually set HDFS configuration parameter like below, it works well
//configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
How can I make my code work? Any inputs would be greatly appreciated.
Regards,
Anand.C
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Instead of setting the xml files programmatically, why not set the
HADOOP_CONF_DIR env variable to '/usr/local/hadoop-1.0.2/conf/'?
That way, you can just create a new Configuration() object and the files
will be loaded for you without any extra work.
On 9 August 2012 09:28, Chandra Mohan, Ananda Vel Murugan <
Ananda.Murugan@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is
> not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my
> code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> >
> >
> > I am trying to add a file to HDFS programmatically.
> >
> >
> >
> > In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> > object as follows
> >
> >
> >
> > Configuration configuration = null;
> >
> > configuration.addResource(new
> > URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.addResource(new
> > Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.reloadConfiguration();
> >
> > fileSystem = FileSystem.get(configuration);
> >
> > System.out.println(fileSystem.getName());
> >
> >
> >
> > This code prints file system as file:/// instead of hdfs://.
> >
> >
> >
> > When I manually set HDFS configuration parameter like below, it works
> well
> >
> >
> >
> > //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >
> >
> >
> > How can I make my code work? Any inputs would be greatly appreciated.
> >
> >
> >
> > Regards,
> >
> > Anand.C
> >
> >
> >
> >
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Instead of setting the xml files programmatically, why not set the
HADOOP_CONF_DIR env variable to '/usr/local/hadoop-1.0.2/conf/'?
That way, you can just create a new Configuration() object and the files
will be loaded for you without any extra work.
On 9 August 2012 09:28, Chandra Mohan, Ananda Vel Murugan <
Ananda.Murugan@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is
> not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my
> code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> >
> >
> > I am trying to add a file to HDFS programmatically.
> >
> >
> >
> > In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> > object as follows
> >
> >
> >
> > Configuration configuration = null;
> >
> > configuration.addResource(new
> > URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.addResource(new
> > Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.reloadConfiguration();
> >
> > fileSystem = FileSystem.get(configuration);
> >
> > System.out.println(fileSystem.getName());
> >
> >
> >
> > This code prints file system as file:/// instead of hdfs://.
> >
> >
> >
> > When I manually set HDFS configuration parameter like below, it works
> well
> >
> >
> >
> > //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >
> >
> >
> > How can I make my code work? Any inputs would be greatly appreciated.
> >
> >
> >
> > Regards,
> >
> > Anand.C
> >
> >
> >
> >
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Instead of setting the xml files programmatically, why not set the
HADOOP_CONF_DIR env variable to '/usr/local/hadoop-1.0.2/conf/'?
That way, you can just create a new Configuration() object and the files
will be loaded for you without any extra work.
On 9 August 2012 09:28, Chandra Mohan, Ananda Vel Murugan <
Ananda.Murugan@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is
> not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my
> code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> >
> >
> > I am trying to add a file to HDFS programmatically.
> >
> >
> >
> > In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> > object as follows
> >
> >
> >
> > Configuration configuration = null;
> >
> > configuration.addResource(new
> > URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.addResource(new
> > Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.reloadConfiguration();
> >
> > fileSystem = FileSystem.get(configuration);
> >
> > System.out.println(fileSystem.getName());
> >
> >
> >
> > This code prints file system as file:/// instead of hdfs://.
> >
> >
> >
> > When I manually set HDFS configuration parameter like below, it works
> well
> >
> >
> >
> > //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >
> >
> >
> > How can I make my code work? Any inputs would be greatly appreciated.
> >
> >
> >
> > Regards,
> >
> > Anand.C
> >
> >
> >
> >
>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Thanks all. I had a typo in core-site.xml path. It works now.
It would be better if conf.addResource throws an exception like FileNotFoundException when it is not able to find the xml file specified.
I like idea of setting HADOOP_CONF_DIR. I will try that too.
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 2:04 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Thanks all. I had a typo in core-site.xml path. It works now.
It would be better if conf.addResource throws an exception like FileNotFoundException when it is not able to find the xml file specified.
I like idea of setting HADOOP_CONF_DIR. I will try that too.
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 2:04 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Thanks all. I had a typo in core-site.xml path. It works now.
It would be better if conf.addResource throws an exception like FileNotFoundException when it is not able to find the xml file specified.
I like idea of setting HADOOP_CONF_DIR. I will try that too.
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 2:04 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Thanks all. I had a typo in core-site.xml path. It works now.
It would be better if conf.addResource throws an exception like FileNotFoundException when it is not able to find the xml file specified.
I like idea of setting HADOOP_CONF_DIR. I will try that too.
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 2:04 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml
On 9 August 2012 09:32, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> > System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:dontariq@gmail.com]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: user@hadoop.apache.org
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> > Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <An...@honeywell.com> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>> System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:dontariq@gmail.com]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: user@hadoop.apache.org
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>> Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <An...@honeywell.com> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml
On 9 August 2012 09:32, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> > System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:dontariq@gmail.com]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: user@hadoop.apache.org
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> > Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <An...@honeywell.com> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml
On 9 August 2012 09:32, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> > System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:dontariq@gmail.com]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: user@hadoop.apache.org
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> > Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <An...@honeywell.com> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml
On 9 August 2012 09:32, Mohammad Tariq <do...@gmail.com> wrote:
> Try this and let me know if it works,
>
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
> conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> > System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:dontariq@gmail.com]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: user@hadoop.apache.org
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> > Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <An...@honeywell.com> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Try this and let me know if it works,
Configuration conf = new Configuration();
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>>
>>
>> I am trying to add a file to HDFS programmatically.
>>
>>
>>
>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>> object as follows
>>
>>
>>
>> Configuration configuration = null;
>>
>> configuration.addResource(new
>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.addResource(new
>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.reloadConfiguration();
>>
>> fileSystem = FileSystem.get(configuration);
>>
>> System.out.println(fileSystem.getName());
>>
>>
>>
>> This code prints file system as file:/// instead of hdfs://.
>>
>>
>>
>> When I manually set HDFS configuration parameter like below, it works well
>>
>>
>>
>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>
>>
>>
>> How can I make my code work? Any inputs would be greatly appreciated.
>>
>>
>>
>> Regards,
>>
>> Anand.C
>>
>>
>>
>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Try this and let me know if it works,
Configuration conf = new Configuration();
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>>
>>
>> I am trying to add a file to HDFS programmatically.
>>
>>
>>
>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>> object as follows
>>
>>
>>
>> Configuration configuration = null;
>>
>> configuration.addResource(new
>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.addResource(new
>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.reloadConfiguration();
>>
>> fileSystem = FileSystem.get(configuration);
>>
>> System.out.println(fileSystem.getName());
>>
>>
>>
>> This code prints file system as file:/// instead of hdfs://.
>>
>>
>>
>> When I manually set HDFS configuration parameter like below, it works well
>>
>>
>>
>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>
>>
>>
>> How can I make my code work? Any inputs would be greatly appreciated.
>>
>>
>>
>> Regards,
>>
>> Anand.C
>>
>>
>>
>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Dave Beech <da...@paraliatech.com>.
Instead of setting the xml files programmatically, why not set the
HADOOP_CONF_DIR env variable to '/usr/local/hadoop-1.0.2/conf/'?
That way, you can just create a new Configuration() object and the files
will be loaded for you without any extra work.
On 9 August 2012 09:28, Chandra Mohan, Ananda Vel Murugan <
Ananda.Murugan@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is
> not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my
> code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
> > Hi,
> >
> >
> >
> > I am trying to add a file to HDFS programmatically.
> >
> >
> >
> > In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> > object as follows
> >
> >
> >
> > Configuration configuration = null;
> >
> > configuration.addResource(new
> > URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.addResource(new
> > Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >
> > configuration.reloadConfiguration();
> >
> > fileSystem = FileSystem.get(configuration);
> >
> > System.out.println(fileSystem.getName());
> >
> >
> >
> > This code prints file system as file:/// instead of hdfs://.
> >
> >
> >
> > When I manually set HDFS configuration parameter like below, it works
> well
> >
> >
> >
> > //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >
> >
> >
> > How can I make my code work? Any inputs would be greatly appreciated.
> >
> >
> >
> > Regards,
> >
> > Anand.C
> >
> >
> >
> >
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Try this and let me know if it works,
Configuration conf = new Configuration();
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>>
>>
>> I am trying to add a file to HDFS programmatically.
>>
>>
>>
>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>> object as follows
>>
>>
>>
>> Configuration configuration = null;
>>
>> configuration.addResource(new
>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.addResource(new
>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.reloadConfiguration();
>>
>> fileSystem = FileSystem.get(configuration);
>>
>> System.out.println(fileSystem.getName());
>>
>>
>>
>> This code prints file system as file:/// instead of hdfs://.
>>
>>
>>
>> When I manually set HDFS configuration parameter like below, it works well
>>
>>
>>
>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>
>>
>>
>> How can I make my code work? Any inputs would be greatly appreciated.
>>
>>
>>
>> Regards,
>>
>> Anand.C
>>
>>
>>
>>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Try this and let me know if it works,
Configuration conf = new Configuration();
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>
> When I add
>
> System.out.println(configuration.get("fs.default.name"));
>
> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>
> Regards,
> Anand.C
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:dontariq@gmail.com]
> Sent: Thursday, August 09, 2012 1:28 PM
> To: user@hadoop.apache.org
> Subject: Re: Question on adding Hadoop XML to Configuration Object
>
> Hello there,
>
> Add the "conf/core-site.xml" file as well.
>
> Regards,
> Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> <An...@honeywell.com> wrote:
>> Hi,
>>
>>
>>
>> I am trying to add a file to HDFS programmatically.
>>
>>
>>
>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>> object as follows
>>
>>
>>
>> Configuration configuration = null;
>>
>> configuration.addResource(new
>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.addResource(new
>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>
>> configuration.reloadConfiguration();
>>
>> fileSystem = FileSystem.get(configuration);
>>
>> System.out.println(fileSystem.getName());
>>
>>
>>
>> This code prints file system as file:/// instead of hdfs://.
>>
>>
>>
>> When I manually set HDFS configuration parameter like below, it works well
>>
>>
>>
>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>
>>
>>
>> How can I make my code work? Any inputs would be greatly appreciated.
>>
>>
>>
>> Regards,
>>
>> Anand.C
>>
>>
>>
>>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Hi,
I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
When I add
System.out.println(configuration.get("fs.default.name"));
I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 1:28 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Hi,
I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
When I add
System.out.println(configuration.get("fs.default.name"));
I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 1:28 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Hi,
I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
When I add
System.out.println(configuration.get("fs.default.name"));
I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 1:28 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
RE: Question on adding Hadoop XML to Configuration Object
Posted by "Chandra Mohan, Ananda Vel Murugan" <An...@honeywell.com>.
Hi,
I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
When I add
System.out.println(configuration.get("fs.default.name"));
I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
Regards,
Anand.C
-----Original Message-----
From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Thursday, August 09, 2012 1:28 PM
To: user@hadoop.apache.org
Subject: Re: Question on adding Hadoop XML to Configuration Object
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>
Re: Question on adding Hadoop XML to Configuration Object
Posted by Mohammad Tariq <do...@gmail.com>.
Hello there,
Add the "conf/core-site.xml" file as well.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
<An...@honeywell.com> wrote:
> Hi,
>
>
>
> I am trying to add a file to HDFS programmatically.
>
>
>
> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
> object as follows
>
>
>
> Configuration configuration = null;
>
> configuration.addResource(new
> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.addResource(new
> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>
> configuration.reloadConfiguration();
>
> fileSystem = FileSystem.get(configuration);
>
> System.out.println(fileSystem.getName());
>
>
>
> This code prints file system as file:/// instead of hdfs://.
>
>
>
> When I manually set HDFS configuration parameter like below, it works well
>
>
>
> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>
>
>
> How can I make my code work? Any inputs would be greatly appreciated.
>
>
>
> Regards,
>
> Anand.C
>
>
>
>