You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by A Geek <dw...@live.com> on 2012/09/18 07:04:05 UTC

Hadoop HDFS and Mapreducer question

Hello All, I'm learning hadoop, hdfs etc and currently tying to solve one issue. Can someone help me how to start attacking the following problem:
I'm trying to come up with some sample code  to store the files  \YEAR\Month\Date\account structure using Hadoop technique s. 
 
 Example: The file will submit to the program as below
 Test_20120917_ACC1.csv and Test_20120916_ACC2.csv
 
 HDFS has to create structure as below
 
 HDFS_HOME\2012\09\17\ACC1\Test_20120917_ACC1.csv 
 HDFS_HOME\2012\09\16\ACC1\Test_20120916_ACC2.csv 

Can someone give me pointers on how to start on this. Highly Appreciated. Thanks for reading the question. 
Thanks,DW

Re: Hadoop HDFS and Mapreducer question

Posted by "Joshi, Rekha" <Re...@intuit.com>.
Refer hadoop put, get syntax for placing input files on hdfs (automate script) and pig dump, store after mapreduce to have your output directory - http://pig.apache.org/docs/r0.9.2/start.html#Pig+Tutorial+Files

Thanks
Rekha

From: A Geek <dw...@live.com>>
Reply-To: <us...@hadoop.apache.org>>
Date: Tue, 18 Sep 2012 05:04:05 +0000
To: <us...@hadoop.apache.org>>
Subject: Hadoop HDFS and Mapreducer question

Hello All,
I'm learning hadoop, hdfs etc and currently tying to solve one issue. Can someone help me how to start attacking the following problem:

I'm trying to come up with some sample code  to store the files  \YEAR\Month\Date\account structure using Hadoop technique s.

Example: The file will submit to the program as below
Test_20120917_ACC1.csv and Test_20120916_ACC2.csv

HDFS has to create structure as below

HDFS_HOME\2012\09\17\ACC1\Test_20120917_ACC1.csv
HDFS_HOME\2012\09\16\ACC1\Test_20120916_ACC2.csv

Can someone give me pointers on how to start on this. Highly Appreciated. Thanks for reading the question.

Thanks,
DW

Re: Hadoop HDFS and Mapreducer question

Posted by "Joshi, Rekha" <Re...@intuit.com>.
Refer hadoop put, get syntax for placing input files on hdfs (automate script) and pig dump, store after mapreduce to have your output directory - http://pig.apache.org/docs/r0.9.2/start.html#Pig+Tutorial+Files

Thanks
Rekha

From: A Geek <dw...@live.com>>
Reply-To: <us...@hadoop.apache.org>>
Date: Tue, 18 Sep 2012 05:04:05 +0000
To: <us...@hadoop.apache.org>>
Subject: Hadoop HDFS and Mapreducer question

Hello All,
I'm learning hadoop, hdfs etc and currently tying to solve one issue. Can someone help me how to start attacking the following problem:

I'm trying to come up with some sample code  to store the files  \YEAR\Month\Date\account structure using Hadoop technique s.

Example: The file will submit to the program as below
Test_20120917_ACC1.csv and Test_20120916_ACC2.csv

HDFS has to create structure as below

HDFS_HOME\2012\09\17\ACC1\Test_20120917_ACC1.csv
HDFS_HOME\2012\09\16\ACC1\Test_20120916_ACC2.csv

Can someone give me pointers on how to start on this. Highly Appreciated. Thanks for reading the question.

Thanks,
DW

Re: Hadoop HDFS and Mapreducer question

Posted by "Joshi, Rekha" <Re...@intuit.com>.
Refer hadoop put, get syntax for placing input files on hdfs (automate script) and pig dump, store after mapreduce to have your output directory - http://pig.apache.org/docs/r0.9.2/start.html#Pig+Tutorial+Files

Thanks
Rekha

From: A Geek <dw...@live.com>>
Reply-To: <us...@hadoop.apache.org>>
Date: Tue, 18 Sep 2012 05:04:05 +0000
To: <us...@hadoop.apache.org>>
Subject: Hadoop HDFS and Mapreducer question

Hello All,
I'm learning hadoop, hdfs etc and currently tying to solve one issue. Can someone help me how to start attacking the following problem:

I'm trying to come up with some sample code  to store the files  \YEAR\Month\Date\account structure using Hadoop technique s.

Example: The file will submit to the program as below
Test_20120917_ACC1.csv and Test_20120916_ACC2.csv

HDFS has to create structure as below

HDFS_HOME\2012\09\17\ACC1\Test_20120917_ACC1.csv
HDFS_HOME\2012\09\16\ACC1\Test_20120916_ACC2.csv

Can someone give me pointers on how to start on this. Highly Appreciated. Thanks for reading the question.

Thanks,
DW

Re: Hadoop HDFS and Mapreducer question

Posted by "Joshi, Rekha" <Re...@intuit.com>.
Refer hadoop put, get syntax for placing input files on hdfs (automate script) and pig dump, store after mapreduce to have your output directory - http://pig.apache.org/docs/r0.9.2/start.html#Pig+Tutorial+Files

Thanks
Rekha

From: A Geek <dw...@live.com>>
Reply-To: <us...@hadoop.apache.org>>
Date: Tue, 18 Sep 2012 05:04:05 +0000
To: <us...@hadoop.apache.org>>
Subject: Hadoop HDFS and Mapreducer question

Hello All,
I'm learning hadoop, hdfs etc and currently tying to solve one issue. Can someone help me how to start attacking the following problem:

I'm trying to come up with some sample code  to store the files  \YEAR\Month\Date\account structure using Hadoop technique s.

Example: The file will submit to the program as below
Test_20120917_ACC1.csv and Test_20120916_ACC2.csv

HDFS has to create structure as below

HDFS_HOME\2012\09\17\ACC1\Test_20120917_ACC1.csv
HDFS_HOME\2012\09\16\ACC1\Test_20120916_ACC2.csv

Can someone give me pointers on how to start on this. Highly Appreciated. Thanks for reading the question.

Thanks,
DW