You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2013/07/12 11:40:16 UTC
how to get hadoop HDFS path?
i want set hdfs path ,AND add the path into hbase,here is my code
Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
System.out.println(":
"+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
htd.setValue("COPROCESSOR$1", path.toString()+"|"
+ TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
and the real value which i find is
hbase(main):012:0> describe 'mytest'
DESCRIPTION
ENABLED
{NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823', FAMILIES => [{N true
AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
REPLICATION_SCOPE => '0', VERSIONS => '3', C
OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
1 row(s) in 0.0930 seconds
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Path path = new
Path("/alex/test.jar<http://192.168.10.22:9000/alex/test.jar>");
//Use relative path here
System.out.println(": "+path.toString()+"|"+TestMyCo.class.
getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
htd.setValue("COPROCESSOR$1", path.toString()+"|"
+ TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
On Fri, Jul 12, 2013 at 2:00 PM, deepak rosario tharigopla <
rozartharigopla@gmail.com> wrote:
> You can get the hdfs file system as follows
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
>
>
> On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
>
>> i want set hdfs path ,AND add the path into hbase,here is my code
>>
>> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
>> System.out.println(":
>> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> htd.setValue("COPROCESSOR$1", path.toString()+"|"
>> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> and the real value which i find is
>>
>> hbase(main):012:0> describe 'mytest'
>> DESCRIPTION
>> ENABLED
>> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
>> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
>> FAMILIES => [{N true
>> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
>> REPLICATION_SCOPE => '0', VERSIONS => '3', C
>> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
>> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
>> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
>> 'true'}]}
>> 1 row(s) in 0.0930 seconds
>>
>
>
>
> --
> Thanks & Regards
> Deepak Rosario Pancras
> *Achiever/Responsibility/Arranger/Maximizer/Harmony*
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Path path = new
Path("/alex/test.jar<http://192.168.10.22:9000/alex/test.jar>");
//Use relative path here
System.out.println(": "+path.toString()+"|"+TestMyCo.class.
getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
htd.setValue("COPROCESSOR$1", path.toString()+"|"
+ TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
On Fri, Jul 12, 2013 at 2:00 PM, deepak rosario tharigopla <
rozartharigopla@gmail.com> wrote:
> You can get the hdfs file system as follows
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
>
>
> On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
>
>> i want set hdfs path ,AND add the path into hbase,here is my code
>>
>> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
>> System.out.println(":
>> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> htd.setValue("COPROCESSOR$1", path.toString()+"|"
>> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> and the real value which i find is
>>
>> hbase(main):012:0> describe 'mytest'
>> DESCRIPTION
>> ENABLED
>> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
>> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
>> FAMILIES => [{N true
>> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
>> REPLICATION_SCOPE => '0', VERSIONS => '3', C
>> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
>> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
>> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
>> 'true'}]}
>> 1 row(s) in 0.0930 seconds
>>
>
>
>
> --
> Thanks & Regards
> Deepak Rosario Pancras
> *Achiever/Responsibility/Arranger/Maximizer/Harmony*
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Path path = new
Path("/alex/test.jar<http://192.168.10.22:9000/alex/test.jar>");
//Use relative path here
System.out.println(": "+path.toString()+"|"+TestMyCo.class.
getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
htd.setValue("COPROCESSOR$1", path.toString()+"|"
+ TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
On Fri, Jul 12, 2013 at 2:00 PM, deepak rosario tharigopla <
rozartharigopla@gmail.com> wrote:
> You can get the hdfs file system as follows
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
>
>
> On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
>
>> i want set hdfs path ,AND add the path into hbase,here is my code
>>
>> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
>> System.out.println(":
>> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> htd.setValue("COPROCESSOR$1", path.toString()+"|"
>> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> and the real value which i find is
>>
>> hbase(main):012:0> describe 'mytest'
>> DESCRIPTION
>> ENABLED
>> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
>> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
>> FAMILIES => [{N true
>> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
>> REPLICATION_SCOPE => '0', VERSIONS => '3', C
>> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
>> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
>> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
>> 'true'}]}
>> 1 row(s) in 0.0930 seconds
>>
>
>
>
> --
> Thanks & Regards
> Deepak Rosario Pancras
> *Achiever/Responsibility/Arranger/Maximizer/Harmony*
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
Path path = new
Path("/alex/test.jar<http://192.168.10.22:9000/alex/test.jar>");
//Use relative path here
System.out.println(": "+path.toString()+"|"+TestMyCo.class.
getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
htd.setValue("COPROCESSOR$1", path.toString()+"|"
+ TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
On Fri, Jul 12, 2013 at 2:00 PM, deepak rosario tharigopla <
rozartharigopla@gmail.com> wrote:
> You can get the hdfs file system as follows
> Configuration conf = new Configuration();
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
> conf.addResource(new
> Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
> FileSystem fs = FileSystem.get(conf);
>
>
> On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
>
>> i want set hdfs path ,AND add the path into hbase,here is my code
>>
>> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
>> System.out.println(":
>> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> htd.setValue("COPROCESSOR$1", path.toString()+"|"
>> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>>
>> and the real value which i find is
>>
>> hbase(main):012:0> describe 'mytest'
>> DESCRIPTION
>> ENABLED
>> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
>> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
>> FAMILIES => [{N true
>> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
>> REPLICATION_SCOPE => '0', VERSIONS => '3', C
>> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
>> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
>> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
>> 'true'}]}
>> 1 row(s) in 0.0930 seconds
>>
>
>
>
> --
> Thanks & Regards
> Deepak Rosario Pancras
> *Achiever/Responsibility/Arranger/Maximizer/Harmony*
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
You can get the hdfs file system as follows
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
> i want set hdfs path ,AND add the path into hbase,here is my code
>
> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
> System.out.println(":
> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> htd.setValue("COPROCESSOR$1", path.toString()+"|"
> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> and the real value which i find is
>
> hbase(main):012:0> describe 'mytest'
> DESCRIPTION
> ENABLED
> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
> FAMILIES => [{N true
> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
> REPLICATION_SCOPE => '0', VERSIONS => '3', C
> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
> 'true'}]}
> 1 row(s) in 0.0930 seconds
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
You can get the hdfs file system as follows
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
> i want set hdfs path ,AND add the path into hbase,here is my code
>
> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
> System.out.println(":
> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> htd.setValue("COPROCESSOR$1", path.toString()+"|"
> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> and the real value which i find is
>
> hbase(main):012:0> describe 'mytest'
> DESCRIPTION
> ENABLED
> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
> FAMILIES => [{N true
> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
> REPLICATION_SCOPE => '0', VERSIONS => '3', C
> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
> 'true'}]}
> 1 row(s) in 0.0930 seconds
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
You can get the hdfs file system as follows
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
> i want set hdfs path ,AND add the path into hbase,here is my code
>
> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
> System.out.println(":
> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> htd.setValue("COPROCESSOR$1", path.toString()+"|"
> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> and the real value which i find is
>
> hbase(main):012:0> describe 'mytest'
> DESCRIPTION
> ENABLED
> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
> FAMILIES => [{N true
> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
> REPLICATION_SCOPE => '0', VERSIONS => '3', C
> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
> 'true'}]}
> 1 row(s) in 0.0930 seconds
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*
Re: how to get hadoop HDFS path?
Posted by deepak rosario tharigopla <ro...@gmail.com>.
You can get the hdfs file system as follows
Configuration conf = new Configuration();
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/core-site.xml"));
conf.addResource(new
Path("/home/dpancras/TradeStation/CassandraPigHadoop/WebContent/WEB-INF/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
On Fri, Jul 12, 2013 at 4:40 AM, ch huang <ju...@gmail.com> wrote:
> i want set hdfs path ,AND add the path into hbase,here is my code
>
> Path path = new Path("hdfs:////192.168.10.22:9000/alex/test.jar");
> System.out.println(":
> "+path.toString()+"|"+TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> htd.setValue("COPROCESSOR$1", path.toString()+"|"
> + TestMyCo.class.getCanonicalName()+"|"+Coprocessor.PRIORITY_USER);
>
> and the real value which i find is
>
> hbase(main):012:0> describe 'mytest'
> DESCRIPTION
> ENABLED
> {NAME => 'mytest', COPROCESSOR$1 => 'hdfs:/
> 192.168.10.22:9000/alex/test.jar|TestMyCo|1073741823<http://192.168.10.22:9000/alex/test.jar%7CTestMyCo%7C1073741823>',
> FAMILIES => [{N true
> AME => 'myfl', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
> REPLICATION_SCOPE => '0', VERSIONS => '3', C
> OMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '6553
> 6', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE =>
> 'true'}]}
> 1 row(s) in 0.0930 seconds
>
--
Thanks & Regards
Deepak Rosario Pancras
*Achiever/Responsibility/Arranger/Maximizer/Harmony*