You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ranjini Rathinam <ra...@gmail.com> on 2014/02/20 13:25:50 UTC
Reg:Hive query with mapreduce
Hi,
How to implement the Hive query such as
select * from table comp;
select empId from comp where sal>12000;
in mapreduce.
Need to use this query in mapreduce code. How to implement the above query
in the code using mapreduce , JAVA.
Please provide the sample code.
Thanks in advance for the support
Regards
Ranjini
Re: Reg:Hive query with mapreduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Assuming you are using TextInputFormat and your data set is comma separated
value , where secondColumn is empId third column is salary, then your
mapfunction would look like this
public class FooMapper extends Mapper<LongWritable,Text,Text,NullWritable>
{
public void map(LongWritable offset, Text empRecord, Context context)
{
String[] splits = empRecord.toString().split(",");
double salary = Double.parseDouble(splits[2]);
if(salary > 120000)
{
context.write(new Text(splits[1],null);
}
}
set the number of reducer tasks to zero.
No of output files would be equal to number of map tasks in this case and
if you want to have single output file then
(1) Set the mapred.min.split.size=<Equal to file size or some bigger value
like Long.MAX_VALUE>>. It will spawn only one mapper task and you will get
one output file
}
Regards,
Som Shekhar Sharma
+91-8197243810
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
Re: Reg:Hive query with mapreduce
Posted by Nitin Pawar <ni...@gmail.com>.
try this
http://ysmart.cse.ohio-state.edu/online.html
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
--
Nitin Pawar
Re: Reg:Hive query with mapreduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Assuming you are using TextInputFormat and your data set is comma separated
value , where secondColumn is empId third column is salary, then your
mapfunction would look like this
public class FooMapper extends Mapper<LongWritable,Text,Text,NullWritable>
{
public void map(LongWritable offset, Text empRecord, Context context)
{
String[] splits = empRecord.toString().split(",");
double salary = Double.parseDouble(splits[2]);
if(salary > 120000)
{
context.write(new Text(splits[1],null);
}
}
set the number of reducer tasks to zero.
No of output files would be equal to number of map tasks in this case and
if you want to have single output file then
(1) Set the mapred.min.split.size=<Equal to file size or some bigger value
like Long.MAX_VALUE>>. It will spawn only one mapper task and you will get
one output file
}
Regards,
Som Shekhar Sharma
+91-8197243810
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
Re: Reg:Hive query with mapreduce
Posted by Nitin Pawar <ni...@gmail.com>.
try this
http://ysmart.cse.ohio-state.edu/online.html
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
--
Nitin Pawar
Re: Reg:Hive query with mapreduce
Posted by Nitin Pawar <ni...@gmail.com>.
try this
http://ysmart.cse.ohio-state.edu/online.html
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
--
Nitin Pawar
Re: Reg:Hive query with mapreduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Assuming you are using TextInputFormat and your data set is comma separated
value , where secondColumn is empId third column is salary, then your
mapfunction would look like this
public class FooMapper extends Mapper<LongWritable,Text,Text,NullWritable>
{
public void map(LongWritable offset, Text empRecord, Context context)
{
String[] splits = empRecord.toString().split(",");
double salary = Double.parseDouble(splits[2]);
if(salary > 120000)
{
context.write(new Text(splits[1],null);
}
}
set the number of reducer tasks to zero.
No of output files would be equal to number of map tasks in this case and
if you want to have single output file then
(1) Set the mapred.min.split.size=<Equal to file size or some bigger value
like Long.MAX_VALUE>>. It will spawn only one mapper task and you will get
one output file
}
Regards,
Som Shekhar Sharma
+91-8197243810
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
Re: Reg:Hive query with mapreduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Assuming you are using TextInputFormat and your data set is comma separated
value , where secondColumn is empId third column is salary, then your
mapfunction would look like this
public class FooMapper extends Mapper<LongWritable,Text,Text,NullWritable>
{
public void map(LongWritable offset, Text empRecord, Context context)
{
String[] splits = empRecord.toString().split(",");
double salary = Double.parseDouble(splits[2]);
if(salary > 120000)
{
context.write(new Text(splits[1],null);
}
}
set the number of reducer tasks to zero.
No of output files would be equal to number of map tasks in this case and
if you want to have single output file then
(1) Set the mapred.min.split.size=<Equal to file size or some bigger value
like Long.MAX_VALUE>>. It will spawn only one mapper task and you will get
one output file
}
Regards,
Som Shekhar Sharma
+91-8197243810
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
Re: Reg:Hive query with mapreduce
Posted by Nitin Pawar <ni...@gmail.com>.
try this
http://ysmart.cse.ohio-state.edu/online.html
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
--
Nitin Pawar
Re: Reg:Hive query with mapreduce
Posted by Nitin Pawar <ni...@gmail.com>.
try this
http://ysmart.cse.ohio-state.edu/online.html
On Thu, Feb 20, 2014 at 5:55 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> How to implement the Hive query such as
>
> select * from table comp;
>
> select empId from comp where sal>12000;
>
> in mapreduce.
>
> Need to use this query in mapreduce code. How to implement the above query
> in the code using mapreduce , JAVA.
>
>
> Please provide the sample code.
>
> Thanks in advance for the support
>
> Regards
>
> Ranjini
>
>
>
>
>
--
Nitin Pawar